ALTER TABLE ADD COLUMN fast default
Attached is a patch for $subject. It's based on Serge Reilau's patch of
about a year ago, taking into account comments made at the time. with
bitrot removed and other enhancements such as documentation.
Essentially it stores a value in pg_attribute that is used when the
stored tuple is missing the attribute. This works unless the default
expression is volatile, in which case a table rewrite is forced as
happens now.
Comments welcome.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v2.patchtext/x-patch; name=fast_default-v2.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3f02202..d5dc14a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1150,6 +1150,18 @@
</row>
<row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
+ <row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
<entry></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242..780bead 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1115,26 +1115,28 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a1a9d99..8188acf 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,105 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrDefault *attrdef;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_defval > 0);
+ Assert(tupleDesc->constr->has_missingval);
+
+ attrdef = tupleDesc->constr->defval;
+
+ /*
+ * Look through the tupledesc's attribute defaults
+ * the on that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_defval - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrdef[missingnum].adnum == attnum)
+ {
+ if (attrdef[missingnum].admissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrdef[missingnum].admissing;
+ }
+ }
+ }
+ Assert(missingnum >= 0);
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ int defattnum;
+ int missingnum;
+ AttrDefault *attrdef;
+
+ if (slot->tts_tupleDescriptor->constr
+ && slot->tts_tupleDescriptor->constr->has_missingval)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_defval - 1;
+ attrdef = slot->tts_tupleDescriptor->constr->defval;
+ }
+ else
+ {
+ missingnum = -1;
+ attrdef = NULL;
+ }
+
+ for (defattnum = tdesc_natts - 1; defattnum >= startAttNum; defattnum--)
+ {
+ if (missingnum >= 0 && attrdef[missingnum].adnum - 1 == defattnum)
+ {
+ if (attrdef[missingnum].admissingNull)
+ {
+ slot->tts_values[defattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[defattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[defattnum] = attrdef[missingnum].admissing;
+ slot->tts_isnull[defattnum] = false;
+ }
+ missingnum--;
+ }
+ else
+ {
+ slot->tts_values[defattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[defattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -133,6 +232,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
}
/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
+/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
*
@@ -172,111 +396,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +421,24 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /* We allow a NULL tupledesc for catalog relations */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
+
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +791,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrDefault *attrdef = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int numdef = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->has_missingval)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ numdef = tupleDesc->constr->num_defval;
+ attrdef = tupleDesc->constr->defval;
+
+ /*
+ * Find the first item in attrdef for which we don't have a value in
+ * the source (i.e. where its adnum is > sourceNatts).
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < numdef
+ && attrdef[firstmissingnum].adnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * Now walk the remaining attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (missingnum = firstmissingnum, attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (missingnum >= numdef)
+ {
+ hasNulls = true;
+ break;
+ }
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrdef[missingnum].adnum == attnum)
+ {
+ if (attrdef[missingnum].admissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrdef[missingnum].admissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrdef[missingnum].admissing);
+ }
+ missingnum++;
+ }
+ else
+ {
+ /*
+ * The missing value attribute number has jumped ahead of the
+ * attribute counter. So there is no missing value for this
+ * attribute, and it must be NULL.
+ */
+ Assert (attrdef[missingnum].adnum > attnum);
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < numdef)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts && missingnum < tupleDesc->constr->num_defval;
+ attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (attrdef[missingnum].adnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrdef[missingnum].admissing,
+ attrdef[missingnum].admissingNull);
+ missingnum++;
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1192,8 +1621,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1691,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1738,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1767,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1790,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca7..e38129a 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -157,6 +158,15 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
{
if (constr->defval[i].adbin)
cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
+ if (constr->defval[i].admissing)
+ {
+ int attnum = constr->defval[i].adnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->defval[i].admissing = datumCopy(constr->defval[i].admissing,
+ attr->attbyval,
+ attr->attlen);
+ }
}
}
@@ -268,6 +278,9 @@ FreeTupleDesc(TupleDesc tupdesc)
{
if (attrdef[i].adbin)
pfree(attrdef[i].adbin);
+ if (attrdef[i].admissing
+ && !TupleDescAttr(tupdesc, attrdef[i].adnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrdef[i].admissing));
}
pfree(attrdef);
}
@@ -338,7 +351,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
{
int i,
j,
- n;
+ n1,
+ n2;
if (tupdesc1->natts != tupdesc2->natts)
return false;
@@ -408,10 +422,9 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (constr1->has_not_null != constr2->has_not_null)
return false;
- n = constr1->num_defval;
- if (n != (int) constr2->num_defval)
- return false;
- for (i = 0; i < n; i++)
+ n1 = constr1->num_defval;
+ n2 = constr2->num_defval;
+ for (i = 0; i < n1; i++)
{
AttrDefault *defval1 = constr1->defval + i;
AttrDefault *defval2 = constr2->defval;
@@ -421,20 +434,24 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
* catalogs in the same order; so use the adnum field to identify
* the matching item to compare.
*/
- for (j = 0; j < n; defval2++, j++)
+ for (j = 0; j < n2; defval2++, j++)
{
if (defval1->adnum == defval2->adnum)
break;
}
- if (j >= n)
+ if (j >= n2)
+ return false;
+ if ((!defval1->adbin && defval2->adbin) ||
+ (defval1->adbin && !defval2->adbin))
return false;
- if (strcmp(defval1->adbin, defval2->adbin) != 0)
+ if (defval1->adbin && defval2->adbin &&
+ strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
- n = constr1->num_check;
- if (n != (int) constr2->num_check)
+ n1 = constr1->num_check;
+ if (n1 != (int) constr2->num_check)
return false;
- for (i = 0; i < n; i++)
+ for (i = 0; i < n1; i++)
{
ConstrCheck *check1 = constr1->check + i;
ConstrCheck *check2 = constr2->check;
@@ -444,7 +461,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
* same order; match them up by name and contents. (The name
* *should* be unique, but...)
*/
- for (j = 0; j < n; check2++, j++)
+ for (j = 0; j < n2; check2++, j++)
{
if (strcmp(check1->ccname, check2->ccname) == 0 &&
strcmp(check1->ccbin, check2->ccbin) == 0 &&
@@ -452,7 +469,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
check1->ccnoinherit == check2->ccnoinherit)
break;
}
- if (j >= n)
+ if (j >= n1)
return false;
}
}
@@ -546,6 +563,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -760,6 +778,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
constr->num_defval = 0;
+ constr->has_missingval = false;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index e481cf3..f49c281 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4444,7 +4444,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9..79b2168 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -447,13 +447,15 @@ sub emit_pgattr_row
attcacheoff => '-1',
atttypmod => '-1',
atthasdef => 'f',
+ atthasmissing => 'f',
attidentity => '',
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
attacl => '_null_',
attoptions => '_null_',
- attfdwoptions => '_null_');
+ attfdwoptions => '_null_',
+ attmissingval => '_null_');
return { %PGATTR_DEFAULTS, %row };
}
@@ -489,6 +491,7 @@ sub emit_schemapg_row
delete $row->{attacl};
delete $row->{attoptions};
delete $row->{attfdwoptions};
+ delete $row->{attmissingval};
# Expand booleans from 'f'/'t' to 'false'/'true'.
# Some values might be other macros (eg FLOAT4PASSBYVAL), don't change.
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4319fc6..7b020a5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -623,6 +627,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -633,6 +638,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1927,7 +1933,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1938,9 +1944,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
false, false);
/*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
+ /*
* Make the pg_attrdef entry.
*/
values[Anum_pg_attrdef_adrelid - 1] = RelationGetRelid(rel);
@@ -1995,7 +2050,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2027,6 +2097,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2179,7 +2260,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2295,7 +2376,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18..6e7ae6a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -353,6 +353,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1577,7 +1578,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 48f1e6e..7e741b6 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -445,7 +445,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 97091dd..7f1c157 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -207,8 +207,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d979ce2..9693923 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -701,6 +701,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4614,7 +4615,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4717,7 +4718,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5333,6 +5334,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5376,6 +5378,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5386,6 +5389,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5452,16 +5464,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -5936,6 +5958,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8066,8 +8089,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9470,7 +9493,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c..417f894 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6c4612d..aab5f2c 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2176,7 +2176,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index dbaa47f..38a6380 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2952,8 +2952,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 51d2c5d..9140e1b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 6a2d5ad..715c6c9 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4403,7 +4403,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -4930,7 +4930,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index e93552a..6b2e614 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index 26c2aed..62af0ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index b1ae9e5..06e2bf4 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 8514c21..36a5236 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1394,7 +1394,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1627,7 +1627,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 12a5f15..d877b52 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -490,6 +491,8 @@ RelationBuildTupleDesc(Relation relation)
TupleConstr *constr;
AttrDefault *attrdef = NULL;
int ndef = 0;
+ int attnum = 0;
+ bool has_missingval = false;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -542,6 +545,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -550,15 +563,72 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
- if (attp->atthasdef)
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
+ if (attp->atthasdef || attp->atthasmissing)
{
+ Datum missingval;
+ bool missingNull;
+
if (attrdef == NULL)
attrdef = (AttrDefault *)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrdef[ndef].admissing = PointerGetDatum(NULL);
+ attrdef[ndef].admissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrdef[ndef].admissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrdef[ndef].admissingNull = false;
+ has_missingval = true;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrdef[ndef].admissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrdef[ndef].admissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrdef[ndef].admissingNull = false;
+ has_missingval = true;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrdef[ndef].admissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrdef[ndef].admissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ has_missingval = true;
+ }
ndef++;
}
need--;
@@ -613,10 +683,14 @@ RelationBuildTupleDesc(Relation relation)
else
constr->defval = attrdef;
constr->num_defval = ndef;
+ constr->has_missingval = has_missingval;
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ constr->has_missingval = false;
+ }
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4090,10 +4164,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4432,7 +4502,8 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred,
+ GetPgIndexDescriptor()))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4727,7 +4798,8 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs,
+ GetPgIndexDescriptor()))
return NIL;
/*
@@ -4790,7 +4862,8 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred,
+ GetPgIndexDescriptor()))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 975c968..ef490d9 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 24a3950..86e876b 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index b0d4c54..fc089a8 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1..87eb398 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -22,7 +23,9 @@
typedef struct attrDefault
{
AttrNumber adnum;
+ bool admissingNull; /* true if missing value is NULL */
char *adbin; /* nodeToString representation of expr */
+ Datum admissing; /* value when attribute is missing */
} AttrDefault;
typedef struct constrCheck
@@ -40,6 +43,7 @@ typedef struct tupleConstr
ConstrCheck *check; /* array */
uint16 num_defval;
uint16 num_check;
+ bool has_missingval;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 0fae022..6292a9d 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8..1a5ff96 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef;
+ /* Has a missing value or not */
+ bool atthasmissing;
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity;
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1];
+
+ /* Missing value for added columns */
+ bytea attmissingval;
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657..ce8458a 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 906dcb8..c02685c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -371,8 +371,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -386,7 +384,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000..e53e491
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e224977..8731ce8 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 9fc5f1a..093edb7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -186,3 +186,4 @@ test: reloptions
test: hash_part
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000..3afc7b8
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
Attached is a patch for $subject. It's based on Serge Reilau's patch of
about a year ago, taking into account comments made at the time. with
bitrot removed and other enhancements such as documentation.
Essentially it stores a value in pg_attribute that is used when the
stored tuple is missing the attribute. This works unless the default
expression is volatile, in which case a table rewrite is forced as
happens now.
I'm quite disturbed both by the sheer invasiveness of this patch
(eg, changing the API of heap_attisnull()) and by the amount of logic
it adds to fundamental tuple-deconstruction code paths. I think
we'll need to ask some hard questions about whether the feature gained
is worth the distributed performance loss that will ensue.
regards, tom lane
On 12/06/2017 10:16 AM, Tom Lane wrote:
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
Attached is a patch for $subject. It's based on Serge Reilau's patch of
about a year ago, taking into account comments made at the time. with
bitrot removed and other enhancements such as documentation.
Essentially it stores a value in pg_attribute that is used when the
stored tuple is missing the attribute. This works unless the default
expression is volatile, in which case a table rewrite is forced as
happens now.I'm quite disturbed both by the sheer invasiveness of this patch
(eg, changing the API of heap_attisnull()) and by the amount of logic
it adds to fundamental tuple-deconstruction code paths. I think
we'll need to ask some hard questions about whether the feature gained
is worth the distributed performance loss that will ensue.
Thanks for prompt comments.
I'm sure we can reduce the footprint some.
w.r.t. heap_attisnull, only a handful of call sites actually need the
new functionality, 8 or possibly 10 (I need to check more on those in
relcache.c). So we could instead of changing the function provide a new
one to be used at those sites. I'll work on that. and submit a new patch
fairly shortly.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On 12/06/2017 10:16 AM, Tom Lane wrote:
I'm quite disturbed both by the sheer invasiveness of this patch
(eg, changing the API of heap_attisnull()) and by the amount of logic
it adds to fundamental tuple-deconstruction code paths. I think
we'll need to ask some hard questions about whether the feature gained
is worth the distributed performance loss that will ensue.
I'm sure we can reduce the footprint some.
w.r.t. heap_attisnull, only a handful of call sites actually need the
new functionality, 8 or possibly 10 (I need to check more on those in
relcache.c). So we could instead of changing the function provide a new
one to be used at those sites. I'll work on that. and submit a new patch
fairly shortly.
Meh, I'm not at all sure that's an improvement. A version of
heap_attisnull that ignores the possibility of a "missing" attribute value
will give the *wrong answer* if the other version should have been used
instead. Furthermore it'd be an attractive nuisance because people would
fail to notice if an existing call were now unsafe. If we're to do this
I think we have no choice but to impose that compatibility break on
third-party code.
In general, you need to be thinking about this in terms of making sure
that it's impossible to accidentally not update code that needs to be
updated. Another example is that I noticed that some places the patch
has s/CreateTupleDescCopy/CreateTupleDescCopyConstr/, evidently because
the former will fail to copy the missing-attribute support data.
I think that's an astonishingly bad idea. There is basically no situation
in which CreateTupleDescCopy defined in that way will ever be safe to use.
The missing-attribute info is now a fundamental part of a tupdesc and it
has to be copied always, just as much as e.g. atttypid.
I would opine that it's a mistake to design TupleDesc as though the
missing-attribute data had anything to do with default values. It may
have originated from the same place but it's a distinct thing. Better
to store it in a separate sub-structure.
regards, tom lane
On 12/06/2017 11:05 AM, Tom Lane wrote:
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On 12/06/2017 10:16 AM, Tom Lane wrote:
I'm quite disturbed both by the sheer invasiveness of this patch
(eg, changing the API of heap_attisnull()) and by the amount of logic
it adds to fundamental tuple-deconstruction code paths. I think
we'll need to ask some hard questions about whether the feature gained
is worth the distributed performance loss that will ensue.I'm sure we can reduce the footprint some.
w.r.t. heap_attisnull, only a handful of call sites actually need the
new functionality, 8 or possibly 10 (I need to check more on those in
relcache.c). So we could instead of changing the function provide a new
one to be used at those sites. I'll work on that. and submit a new patch
fairly shortly.Meh, I'm not at all sure that's an improvement. A version of
heap_attisnull that ignores the possibility of a "missing" attribute value
will give the *wrong answer* if the other version should have been used
instead. Furthermore it'd be an attractive nuisance because people would
fail to notice if an existing call were now unsafe. If we're to do this
I think we have no choice but to impose that compatibility break on
third-party code.
Fair point.
In general, you need to be thinking about this in terms of making sure
that it's impossible to accidentally not update code that needs to be
updated. Another example is that I noticed that some places the patch
has s/CreateTupleDescCopy/CreateTupleDescCopyConstr/, evidently because
the former will fail to copy the missing-attribute support data.
I think that's an astonishingly bad idea. There is basically no situation
in which CreateTupleDescCopy defined in that way will ever be safe to use.
The missing-attribute info is now a fundamental part of a tupdesc and it
has to be copied always, just as much as e.g. atttypid.I would opine that it's a mistake to design TupleDesc as though the
missing-attribute data had anything to do with default values. It may
have originated from the same place but it's a distinct thing. Better
to store it in a separate sub-structure.
OK, will work on all that. Thanks again.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 12/06/2017 11:26 AM, Andrew Dunstan wrote:
In general, you need to be thinking about this in terms of making sure
that it's impossible to accidentally not update code that needs to be
updated. Another example is that I noticed that some places the patch
has s/CreateTupleDescCopy/CreateTupleDescCopyConstr/, evidently because
the former will fail to copy the missing-attribute support data.
I think that's an astonishingly bad idea. There is basically no situation
in which CreateTupleDescCopy defined in that way will ever be safe to use.
The missing-attribute info is now a fundamental part of a tupdesc and it
has to be copied always, just as much as e.g. atttypid.I would opine that it's a mistake to design TupleDesc as though the
missing-attribute data had anything to do with default values. It may
have originated from the same place but it's a distinct thing. Better
to store it in a separate sub-structure.OK, will work on all that. Thanks again.
Looking closer at this. First, there is exactly one place where the
patch does s/CreateTupleDescCopy/CreateTupleDescCopyConstr/.
making this like atttypid suggests that it belongs right outside the
TupleConstr structure. But then it seems to me that the change could
well end up being a darn site more invasive. For example, should we be
passing the number of missing values to CreateTemplateTupleDesc and
CreateTupleDesc? I'm happy to do whatever work is required, but I want
to have a firm understanding of the design before I spend lots of time
cutting code. Once I understand how tupdesc.h should look I should be good.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 12/12/2017 02:13 PM, Andrew Dunstan wrote:
On 12/06/2017 11:26 AM, Andrew Dunstan wrote:
In general, you need to be thinking about this in terms of making sure
that it's impossible to accidentally not update code that needs to be
updated. Another example is that I noticed that some places the patch
has s/CreateTupleDescCopy/CreateTupleDescCopyConstr/, evidently because
the former will fail to copy the missing-attribute support data.
I think that's an astonishingly bad idea. There is basically no situation
in which CreateTupleDescCopy defined in that way will ever be safe to use.
The missing-attribute info is now a fundamental part of a tupdesc and it
has to be copied always, just as much as e.g. atttypid.I would opine that it's a mistake to design TupleDesc as though the
missing-attribute data had anything to do with default values. It may
have originated from the same place but it's a distinct thing. Better
to store it in a separate sub-structure.OK, will work on all that. Thanks again.
Looking closer at this. First, there is exactly one place where the
patch does s/CreateTupleDescCopy/CreateTupleDescCopyConstr/.making this like atttypid suggests that it belongs right outside the
TupleConstr structure. But then it seems to me that the change could
well end up being a darn site more invasive. For example, should we be
passing the number of missing values to CreateTemplateTupleDesc and
CreateTupleDesc? I'm happy to do whatever work is required, but I want
to have a firm understanding of the design before I spend lots of time
cutting code. Once I understand how tupdesc.h should look I should be good.
Here is a new version of the patch. It separates the missing values
constructs and logic from the default values constructs and logic. The
missing values now sit alongside the default values in tupleConstr. In
some places the separation makes the logic a good bit cleaner.
Some of the logic is also reworked to remove an assumption that the
missing values structure is populated in attnum order, Also code to
support the missing stuctures is added to equalTupleDescs.
We still have that one extra place where we need to call
CreateTupleDescCopyConstr instead of CreateTupleDescCopy. I haven't seen
an easy way around that.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v4.patchtext/x-patch; name=fast_default-v4.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3f02202..d5dc14a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1150,6 +1150,18 @@
</row>
<row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
+ <row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
<entry></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242..780bead 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1115,26 +1115,28 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a1a9d99..6aab16f 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -133,6 +243,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
}
/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
+/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
*
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,27 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
+
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +805,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1192,8 +1635,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1705,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1752,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1781,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1804,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca7..795249f 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -160,6 +161,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -271,6 +290,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -431,6 +464,28 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Same logic as for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ /*
+ * can't check the actual values cheaply here
+ */
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -546,6 +601,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -759,7 +815,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index e481cf3..f49c281 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4444,7 +4444,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 5b5b04f..ee9da4a 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -450,13 +450,15 @@ sub emit_pgattr_row
attcacheoff => '-1',
atttypmod => '-1',
atthasdef => 'f',
+ atthasmissing => 'f',
attidentity => '',
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
attacl => '_null_',
attoptions => '_null_',
- attfdwoptions => '_null_');
+ attfdwoptions => '_null_',
+ attmissingval => '_null_');
return { %PGATTR_DEFAULTS, %row };
}
@@ -492,6 +494,7 @@ sub emit_schemapg_row
delete $row->{attacl};
delete $row->{attoptions};
delete $row->{attfdwoptions};
+ delete $row->{attmissingval};
# Expand booleans from 'f'/'t' to 'false'/'true'.
# Some values might be other macros (eg FLOAT4PASSBYVAL), don't change.
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4319fc6..7b020a5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -623,6 +627,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -633,6 +638,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1927,7 +1933,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1938,9 +1944,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
false, false);
/*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
+ /*
* Make the pg_attrdef entry.
*/
values[Anum_pg_attrdef_adrelid - 1] = RelationGetRelid(rel);
@@ -1995,7 +2050,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2027,6 +2097,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2179,7 +2260,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2295,7 +2376,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18..6e7ae6a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -353,6 +353,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1577,7 +1578,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 1c5669a..d0f26e3 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -445,7 +445,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 97091dd..7f1c157 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -207,8 +207,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d979ce2..9693923 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -701,6 +701,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4614,7 +4615,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4717,7 +4718,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5333,6 +5334,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5376,6 +5378,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5386,6 +5389,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5452,16 +5464,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -5936,6 +5958,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8066,8 +8089,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9470,7 +9493,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c..417f894 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index fa4ab30..db96811 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2212,7 +2212,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index dbaa47f..38a6380 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2952,8 +2952,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 51d2c5d..9140e1b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 9ca384d..14aec10 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4416,7 +4416,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -4943,7 +4943,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index e93552a..6b2e614 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index 26c2aed..62af0ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index b1ae9e5..06e2bf4 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 8514c21..36a5236 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1394,7 +1394,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1627,7 +1627,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1d0cc6c..072bb3b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -488,7 +489,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -541,6 +545,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -549,6 +563,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -556,10 +574,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -600,7 +679,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -615,7 +695,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4045,10 +4137,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4387,7 +4475,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4682,7 +4770,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4745,7 +4833,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 975c968..ef490d9 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 24a3950..86e876b 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index b0d4c54..fc089a8 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1..d488320 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 0fae022..6292a9d 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8..1a5ff96 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef;
+ /* Has a missing value or not */
+ bool atthasmissing;
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity;
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1];
+
+ /* Missing value for added columns */
+ bytea attmissingval;
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657..ce8458a 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 906dcb8..c02685c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -371,8 +371,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -386,7 +384,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000..e53e491
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e224977..8731ce8 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 9fc5f1a..093edb7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -186,3 +186,4 @@ test: reloptions
test: hash_part
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000..3afc7b8
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 12/31/2017 06:48 PM, Andrew Dunstan wrote:
Here is a new version of the patch. It separates the missing values
constructs and logic from the default values constructs and logic. The
missing values now sit alongside the default values in tupleConstr. In
some places the separation makes the logic a good bit cleaner.Some of the logic is also reworked to remove an assumption that the
missing values structure is populated in attnum order, Also code to
support the missing stuctures is added to equalTupleDescs.We still have that one extra place where we need to call
CreateTupleDescCopyConstr instead of CreateTupleDescCopy. I haven't seen
an easy way around that.
New version of the patch that fills in the remaining piece in
equalTupleDescs.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v5.patchtext/x-patch; name=fast_default-v5.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3f02202..d5dc14a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1150,6 +1150,18 @@
</row>
<row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
+ <row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
<entry></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242..780bead 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1115,26 +1115,28 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a1a9d99..6aab16f 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -133,6 +243,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
}
/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
+/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
*
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,27 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
+
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +805,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1192,8 +1635,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1705,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1752,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1781,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1804,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca7..d604a1a 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -160,6 +161,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -271,6 +290,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -431,6 +464,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -546,6 +608,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -759,7 +822,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index e481cf3..f49c281 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4444,7 +4444,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 5b5b04f..ee9da4a 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -450,13 +450,15 @@ sub emit_pgattr_row
attcacheoff => '-1',
atttypmod => '-1',
atthasdef => 'f',
+ atthasmissing => 'f',
attidentity => '',
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
attacl => '_null_',
attoptions => '_null_',
- attfdwoptions => '_null_');
+ attfdwoptions => '_null_',
+ attmissingval => '_null_');
return { %PGATTR_DEFAULTS, %row };
}
@@ -492,6 +494,7 @@ sub emit_schemapg_row
delete $row->{attacl};
delete $row->{attoptions};
delete $row->{attfdwoptions};
+ delete $row->{attmissingval};
# Expand booleans from 'f'/'t' to 'false'/'true'.
# Some values might be other macros (eg FLOAT4PASSBYVAL), don't change.
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4319fc6..7b020a5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -623,6 +627,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -633,6 +638,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1927,7 +1933,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1938,9 +1944,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
false, false);
/*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
+ /*
* Make the pg_attrdef entry.
*/
values[Anum_pg_attrdef_adrelid - 1] = RelationGetRelid(rel);
@@ -1995,7 +2050,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2027,6 +2097,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2179,7 +2260,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2295,7 +2376,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18..6e7ae6a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -353,6 +353,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1577,7 +1578,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 1c5669a..d0f26e3 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -445,7 +445,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 97091dd..7f1c157 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -207,8 +207,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d979ce2..9693923 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -701,6 +701,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4614,7 +4615,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4717,7 +4718,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5333,6 +5334,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5376,6 +5378,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5386,6 +5389,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5452,16 +5464,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -5936,6 +5958,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8066,8 +8089,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9470,7 +9493,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c..417f894 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index fa4ab30..db96811 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2212,7 +2212,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index dbaa47f..38a6380 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2952,8 +2952,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 51d2c5d..9140e1b 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 9ca384d..14aec10 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4416,7 +4416,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -4943,7 +4943,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index e93552a..6b2e614 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index 26c2aed..62af0ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index b1ae9e5..06e2bf4 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 8514c21..36a5236 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1394,7 +1394,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1627,7 +1627,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1d0cc6c..072bb3b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -488,7 +489,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -541,6 +545,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -549,6 +563,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -556,10 +574,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -600,7 +679,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -615,7 +695,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4045,10 +4137,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4387,7 +4475,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4682,7 +4770,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4745,7 +4833,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 975c968..ef490d9 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 24a3950..86e876b 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index b0d4c54..fc089a8 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1..d488320 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 0fae022..6292a9d 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8..1a5ff96 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef;
+ /* Has a missing value or not */
+ bool atthasmissing;
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity;
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1];
+
+ /* Missing value for added columns */
+ bytea attmissingval;
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657..ce8458a 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 906dcb8..c02685c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -371,8 +371,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -386,7 +384,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000..e53e491
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e224977..8731ce8 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 9fc5f1a..093edb7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -186,3 +186,4 @@ test: reloptions
test: hash_part
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000..3afc7b8
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 2 January 2018 at 05:01, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
New version of the patch that fills in the remaining piece in
equalTupleDescs.
This no longer applies to current master. Can send an updated patch?
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 01/16/2018 12:13 AM, David Rowley wrote:
On 2 January 2018 at 05:01, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:New version of the patch that fills in the remaining piece in
equalTupleDescs.This no longer applies to current master. Can send an updated patch?
Yeah, got caught by the bki/pg_attribute changes on Friday. here's an
updated version. Thanks for looking.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v6.patchtext/x-patch; name=fast_default-v6.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3f02202..d5dc14a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1150,6 +1150,18 @@
</row>
<row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
+ <row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
<entry></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242..780bead 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1115,26 +1115,28 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251..5ea3f1c 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -133,6 +243,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
}
/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
+/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
*
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,27 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
+
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +805,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1192,8 +1635,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1705,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1752,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1781,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1804,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f4423..8538359 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index fac8061..8640103 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4444,7 +4444,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02..3b0be39 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 089b796..b9133c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -623,6 +627,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -633,6 +638,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1927,7 +1933,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1938,9 +1944,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
false, false);
/*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
+ /*
* Make the pg_attrdef entry.
*/
values[Anum_pg_attrdef_adrelid - 1] = RelationGetRelid(rel);
@@ -1995,7 +2050,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2027,6 +2097,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2179,7 +2260,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2295,7 +2376,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 330488b..57a6a6b 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -353,6 +353,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1577,7 +1578,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index eb73299..0b481a9 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -445,7 +445,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9e6ba92..2152bb4 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -207,8 +207,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f2a928b..36cfed0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -700,6 +700,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4613,7 +4614,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4716,7 +4717,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5332,6 +5333,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5375,6 +5377,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5385,6 +5388,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5451,16 +5463,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -5935,6 +5957,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8005,8 +8028,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9411,7 +9434,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index a40b3cf..6492438 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index f646fd9..d49cf07 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2453,7 +2453,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 16822e9..efbaec1 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2967,8 +2967,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5df89e4..68d67e6 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce..2fefa79 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 32e3798..9bfcb84 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3..7eb981a 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d..30b7e62 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 9cdbb06..1b8560c 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1394,7 +1394,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1627,7 +1627,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 00ba33b..907d75b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -488,7 +489,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -541,6 +545,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -549,6 +563,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -556,10 +574,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -600,7 +679,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -615,7 +695,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4045,10 +4137,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4387,7 +4475,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4682,7 +4770,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4745,7 +4833,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 8968a9f..dd3bd5c 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bf..d5580de 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815..fb1ad17 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efba..e62100d 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63c..5869a48 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383..830b1cf 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index e704943..6ad6862 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 906dcb8..c02685c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -371,8 +371,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -386,7 +384,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000..e53e491
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e224977..8731ce8 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 9fc5f1a..093edb7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -186,3 +186,4 @@ test: reloptions
test: hash_part
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000..3afc7b8
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On Wed, Jan 17, 2018 at 2:21 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
Yeah, got caught by the bki/pg_attribute changes on Friday. here's an
updated version. Thanks for looking.
A boring semi-automated update: this no long compiles, because
8561e4840c8 added a new call to heap_attisnull(). Pretty sure it just
needs NULL in the third argument.
--
Thomas Munro
http://www.enterprisedb.com
On Thu, Jan 25, 2018 at 6:10 PM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Wed, Jan 17, 2018 at 2:21 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, got caught by the bki/pg_attribute changes on Friday. here's an
updated version. Thanks for looking.A boring semi-automated update: this no long compiles, because
8561e4840c8 added a new call to heap_attisnull(). Pretty sure it just
needs NULL in the third argument.
Yeah, thanks. revised patch attached
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v7.patchapplication/octet-stream; name=fast_default-v7.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..99f776e7c1 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 286c7a8589..80a52eda2c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1141,26 +1141,28 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..5ea3f1c958 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +242,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,27 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
+
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +805,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1192,8 +1635,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1705,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1752,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1781,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1804,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..85383592ef 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02303..3b0be39101 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 774c07b03a..4db704ad5c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2098,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2180,7 +2261,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 849a469127..67c61820f4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -370,6 +370,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1626,7 +1627,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 1701548d84..b9cc736adf 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index df87dfeb54..3a4f56fc48 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2321,7 +2321,7 @@ ExecuteCallStmt(ParseState *pstate, CallStmt *stmt, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index a9461a4b06..f07587a04d 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -214,8 +214,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2e768dd5e4..a545aebec2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4678,7 +4679,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4781,7 +4782,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5400,6 +5401,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5443,6 +5445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5453,6 +5456,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5519,16 +5531,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6003,6 +6025,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8078,8 +8101,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9485,7 +9508,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 74eb430f96..4af887b992 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index f646fd9c51..d49cf07c1a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2453,7 +2453,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 410921cc40..af70440642 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2967,8 +2967,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5df89e419c..68d67e68b2 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 32e3798972..9bfcb840bd 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c5f5a1ca3f..10c47d41da 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1396,7 +1396,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1629,7 +1629,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c081b88b73..a3b55847b0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +568,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +579,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -605,7 +684,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +700,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4054,10 +4146,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4396,7 +4484,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4691,7 +4779,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4754,7 +4842,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..e62100d253 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..830b1cf0c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..e53e491007
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..13b707e295 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..3afc7b82b6
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
Hi,
On 2018-01-26 10:53:12 +1030, Andrew Dunstan wrote:
Yeah, thanks. revised patch attached
Given that this patch touches code that's a huge bottleneck in a lot of
cases, I think this needs benchmarks that heavily exercises tuple
deforming.
Greetings,
Andres Freund
On 02/01/2018 02:54 PM, Andres Freund wrote:
Hi,
On 2018-01-26 10:53:12 +1030, Andrew Dunstan wrote:
Yeah, thanks. revised patch attached
Given that this patch touches code that's a huge bottleneck in a lot of
cases, I think this needs benchmarks that heavily exercises tuple
deforming.
That's a reasonable request, I guess, and I tried to collect such data
today. I measured two cases:
1) Table with 1000 columns and 64 rows, when there were no ALTER TABLE
adding columns with 'fast' defaults. This is meant to measure the best
case, with minimal impact from the patch. See the create.sql script
attached to this message, and the query.sql which was used for tests
using pgbench like this:
pgbench -n -f q.sql -T 15 test
after 100 runs, the results (tps) look like this:
min max median
--------------------------------------
master 1827 1873 1860
patched 2023 2066 2056
That is, the patch apparently improves the performance by about 10%
(according to perf profiles this is due to slot_deform_tuple getting
cheaper).
So this case seems fine.
2) Table with 64 rows and 1000 columns, all added by ALTER TABLE with
fast default without rewrite. See create-alter.sql.
Using the same query.sql as before, this shold significant drop to only
about 40 tps (from ~2000 tps for master). The profiles something like this:
+ 98.87% 98.87% postgres [.] slot_getmissingattrs
+ 98.77% 0.00% postgres [.] PortalRun
+ 98.77% 0.00% postgres [.] ExecAgg
+ 98.74% 0.01% postgres [.] ExecInterpExpr
which is kinda understandable, although the 2000 to 40 tps seems like a
pretty significant drop. But then again, this case is constructed like a
fairly extreme corner case.
However, there seems to be some sort of bug, because when I did VACUUM
FULL - ideally this would replace the "missing" default values with
actual values stored in the heap rows, eliminating the performance
impact. But the default values got lost and replaced by NULL values,
which seems like a clear data loss scenario.
I'm not quite sure what's wrong, but try this:
\i create-alter.sql
-- this returns 64, which is correct
SELECT COUNT(*) FROM t;
-- this actually retuns 64 rows with values "1"
SELECT c1000 FROM t;
-- this returns 63, which is incorrect (should be 64)
SELECT count(c1000) FROM t;
VACUUM FULL t;
-- suddenly we only get NULL values for all 64 rows
SELECT c1000 FROM t;
-- and now we got 0 (instead of 64)
SELECT count(c1000) FROM t;
regard
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)
*** /home/travis/build/postgresql-cfbot/postgresql/src/test/regress/expected/identity.out
2018-02-04 00:56:12.146303616 +0000
--- /home/travis/build/postgresql-cfbot/postgresql/src/test/regress/results/identity.out
2018-02-04 01:05:55.217932032 +0000
***************
*** 257,268 ****
INSERT INTO itest13 VALUES (1), (2), (3);
-- add column to populated table
ALTER TABLE itest13 ADD COLUMN c int GENERATED BY DEFAULT AS IDENTITY;
SELECT * FROM itest13;
! a | b | c
! ---+---+---
! 1 | 1 | 1
! 2 | 2 | 2
! 3 | 3 | 3
(3 rows)
-- various ALTER COLUMN tests
--- 257,269 ----
INSERT INTO itest13 VALUES (1), (2), (3);
-- add column to populated table
ALTER TABLE itest13 ADD COLUMN c int GENERATED BY DEFAULT AS IDENTITY;
+ ERROR: column "c" contains null values
SELECT * FROM itest13;
! a | b
! ---+---
! 1 | 1
! 2 | 2
! 3 | 3
(3 rows)
-- various ALTER COLUMN tests
--
Thomas Munro
http://www.enterprisedb.com
On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:
On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)
Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Feb 5, 2018 at 7:49 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v8.patchapplication/octet-stream; name=fast_default-v8.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..99f776e7c1 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2b514b7606..64e5e757d4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1172,26 +1172,28 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..1ac70b9c4e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +242,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,25 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +803,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1453,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as null, or a missing value if there is one.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1630,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1700,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1747,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1776,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1799,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..85383592ef 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02303..3b0be39101 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 0f34f5381a..7cd36cdbe9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2098,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2180,7 +2261,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index f2cb6d7fb8..ec1f780ef6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1628,7 +1629,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..aa654067ba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index a483714766..b8b5b579e5 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2266,7 +2266,7 @@ ExecuteCallStmt(ParseState *pstate, CallStmt *stmt, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 7c46613215..757bf6d35f 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -214,8 +214,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 89454d8e80..1193710822 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4678,7 +4679,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4781,7 +4782,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5400,6 +5401,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5443,6 +5445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5453,6 +5456,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5498,6 +5510,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5533,16 +5548,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6017,6 +6042,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8092,8 +8118,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9499,7 +9525,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index f646fd9c51..d49cf07c1a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2453,7 +2453,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5d3e923cca..efdbb31fb2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5df89e419c..68d67e68b2 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..84c11242bf 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 28767a129a..00b1fb4950 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1396,7 +1396,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1629,7 +1629,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d5cc246156..7dfd4fa74b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +568,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +579,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -605,7 +684,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +700,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4151,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4489,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4784,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4847,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..e62100d253 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..830b1cf0c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..e53e491007
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 201 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..13b707e295 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..3afc7b82b6
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 09/02/18 06:24, Andrew Dunstan wrote:
On Mon, Feb 5, 2018 at 7:49 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.
I see the patch does not update the ALTER TABLE docs section which
discusses table rewrites and it seems like it should.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Sun, Feb 11, 2018 at 2:50 PM, Petr Jelinek
<petr.jelinek@2ndquadrant.com> wrote:
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.I see the patch does not update the ALTER TABLE docs section which
discusses table rewrites and it seems like it should.
Umm it changes the second and third paras of the Notes section, which
refer to rewrites. Not sure what else it should change.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Fri, Feb 9, 2018 at 3:54 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Mon, Feb 5, 2018 at 7:49 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.
This version fixes that issue. Thanks to Tomas for his help in finding
where the problem was. There are now no outstanding issues that I'm
aware of.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v9.patchapplication/octet-stream; name=fast_default-v9.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..99f776e7c1 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2b514b7606..64e5e757d4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1172,26 +1172,28 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..1ac70b9c4e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,116 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ /*
+ * Loop through the list of missing attributes (if any) looking for
+ * a corresponding entry. Don't assume that the missing values
+ * are in attribute order.
+ */
+ for (i = missingnum; i >= 0; i--)
+ {
+ Assert(attrmiss != NULL);
+ if (attrmiss[i].amnum - 1 == missattnum)
+ {
+ if (attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ else
+ {
+ slot->tts_values[missattnum] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ break;
+ }
+ }
+ /*
+ * If there is no corresponding missing entry use NULL.
+ */
+ if (i < 0)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +242,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +407,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +432,25 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +803,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1453,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as null, or a missing value if there is one.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1630,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1700,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1747,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1776,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1799,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..85383592ef 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02303..3b0be39101 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..35871eadab 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2098,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2180,7 +2261,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index f2cb6d7fb8..ec1f780ef6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1628,7 +1629,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..aa654067ba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index a027b19744..7228aa0d49 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2275,7 +2275,7 @@ ExecuteCallStmt(ParseState *pstate, CallStmt *stmt, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 7c46613215..757bf6d35f 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -214,8 +214,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 89454d8e80..1193710822 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4678,7 +4679,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4781,7 +4782,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5400,6 +5401,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5443,6 +5445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5453,6 +5456,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5498,6 +5510,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5533,16 +5548,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6017,6 +6042,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8092,8 +8118,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9499,7 +9525,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index f646fd9c51..d49cf07c1a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2453,7 +2453,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5d3e923cca..efdbb31fb2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5df89e419c..68d67e68b2 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 50b6edce63..9832b2477d 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -550,6 +550,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..84c11242bf 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3bb468bdad..465d574510 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1396,7 +1396,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1629,7 +1629,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..046e9d012b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +568,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +579,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -605,7 +684,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +700,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4151,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4489,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4784,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4847,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..e62100d253 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..830b1cf0c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..eecec9e65e
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..13b707e295 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..3afc7b82b6
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On Tue, Feb 13, 2018 at 6:28 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Fri, Feb 9, 2018 at 3:54 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:49 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.This version fixes that issue. Thanks to Tomas for his help in finding
where the problem was. There are now no outstanding issues that I'm
aware of.
The attached version largely fixes with the performance degradation
that Tomas Vondra reported, replacing an O(N^2) piece of code in
slot_getmissingattrs() by one that is O(N).
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v10.patchapplication/octet-stream; name=fast_default-v10.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..99f776e7c1 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the column
+ is entirely missing from the row, as happens when the column is added after
+ the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2b514b7606..64e5e757d4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1172,26 +1172,28 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default
+ is evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for
+ all existing rows. If no <literal>DEFAULT</literal> is specified
+ NULL is used. In neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of
+ an existing column will require the entire table and its indexes to be
+ rewritten. As an exception when changing the type of an existing column,
+ if the <literal>USING</literal> clause does not change the column
+ contents and the old type is either binary coercible to the new type or
+ an unconstrained domain over the new type, a table rewrite is not needed;
+ but any indexes on the affected columns must still be rebuilt. Adding or
+ removing a system <literal>oid</literal> column also requires rewriting
+ the entire table.
+ Table and/or index rebuilds may take a significant amount of time
+ for a large table; and will temporarily require as much as double the disk
+ space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..7aca79f4a6 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,99 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum)
+{
+ int tdesc_natts = slot->tts_tupleDescriptor->natts;
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
+
+ /* initialize all the missing sttributes to null */
+ for (missattnum = tdesc_natts - 1; missattnum >= startAttNum; missattnum--)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+
+ /* now replace any we have a non-null missing value for */
+ for (i = missingnum; i >= 0; i--)
+ {
+ int missattn = attrmiss[i].amnum - 1;
+ if (missattn >= startAttNum && !attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattn] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattn] = false;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +225,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +390,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +415,25 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
+ }
if (attnum > 0)
{
@@ -649,6 +786,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1436,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as null, or a missing value if there is one.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1613,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1683,13 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,12 +1730,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1759,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1782,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..85383592ef 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02303..3b0be39101 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..35871eadab 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2098,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2180,7 +2261,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index f2cb6d7fb8..ec1f780ef6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1628,7 +1629,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..aa654067ba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index a027b19744..7228aa0d49 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2275,7 +2275,7 @@ ExecuteCallStmt(ParseState *pstate, CallStmt *stmt, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 7c46613215..757bf6d35f 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -214,8 +214,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 89454d8e80..1193710822 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4678,7 +4679,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4781,7 +4782,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5400,6 +5401,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5443,6 +5445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5453,6 +5456,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5498,6 +5510,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5533,16 +5548,26 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
/*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
*/
- tab->new_notnull |= colDef->is_not_null;
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, tell Phase 3 it needs to test
+ * that. (Note we don't do this for an OID column. OID will be
+ * marked not null, but since it's filled specially, there's no
+ * need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6017,6 +6042,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8092,8 +8118,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9499,7 +9525,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 9c6c2b02e9..fba620a473 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2454,7 +2454,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 5d3e923cca..efdbb31fb2 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,16 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) < RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 5df89e419c..68d67e68b2 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -589,7 +589,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -627,7 +635,22 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 50b6edce63..9832b2477d 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -550,6 +550,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..84c11242bf 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,7 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index ba9fab4582..1cd5835686 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1396,7 +1396,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1629,7 +1629,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..046e9d012b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,16 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
+
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +568,10 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /*
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
+ */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +579,71 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -605,7 +684,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +700,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4151,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4489,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4784,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4847,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..e62100d253 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..830b1cf0c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,30 +197,31 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Natts_pg_attribute 24
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..eecec9e65e
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..13b707e295 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -111,7 +111,7 @@ test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combo
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid conversion truncate alter_table sequence polymorphism rowtypes returning largeobject with xml fast_default
# ----------
# Another group of parallel tests
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..3afc7b82b6
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of dfferent datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 02/16/2018 10:46 PM, Andrew Dunstan wrote:
On Tue, Feb 13, 2018 at 6:28 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Fri, Feb 9, 2018 at 3:54 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:49 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Mon, Feb 5, 2018 at 7:19 AM, Thomas Munro
<thomas.munro@enterprisedb.com> wrote:On Fri, Jan 26, 2018 at 1:23 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Yeah, thanks. revised patch attached
FYI the identity regression test started failing recently with this
patch applied (maybe due to commit
533c5d8bddf0feb1785b3da17c0d17feeaac76d8?)Thanks. Probably the same bug Tomas Vondra found a few days ago. I'm on it.
Here's a version that fixes the above issue and also the issue with
VACUUM that Tomas Vondra reported. I'm still working on the issue with
aggregates that Tomas also reported.This version fixes that issue. Thanks to Tomas for his help in finding
where the problem was. There are now no outstanding issues that I'm
aware of.The attached version largely fixes with the performance degradation
that Tomas Vondra reported, replacing an O(N^2) piece of code in
slot_getmissingattrs() by one that is O(N).
Seems fine to me - for comparison, numbers for master and v9/v10 patch
versions look like this:
create.sql create-alter.sql
-----------------------------------------
master 1120 1320
v9 1100 50
v10 1130 1370
This particular machine has a laptop-class CPU, so the timings are
somewhat noisy - which explains the small tps differences.
What I find interesting is that if I do VACUUM FULL after running the
SQL script, I get this:
create.sql create-alter.sql
-----------------------------------------
master 1120 1450
v9 1100 1320
v10 1100 1450
That is, the throughput for create-alter case jumps up by about 100 tps,
for some reason. Not sure why, though ...
Anyway, I consider the performance to be OK. But perhaps Andres could
comment on this too, as he requested the benchmarks.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 17 February 2018 at 10:46, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
The attached version largely fixes with the performance degradation
that Tomas Vondra reported, replacing an O(N^2) piece of code in
slot_getmissingattrs() by one that is O(N).
I've looked over this patch and had a play around with it.
Just a couple of things that I noticed while reading:
1. I believe "its" should be "it's" here:
+ /*
+ * Yes, and its of the by-value kind Copy in the Datum
+ */
copy does not need an upper case 'C'. There should also be a comma after "kind"
There's a similar problem with the following:
+ /*
+ * Yes, and its fixed length Copy it out and have teh Datum
+ * point to it.
+ */
there should be a comma after "length" too.
Also, "teh" ...
2. "dfferent" -> "different"
+-- Test a large sample of dfferent datatypes
I'll try to spend some time reviewing the code in detail soon.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Sun, Feb 18, 2018 at 2:52 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:
On 17 February 2018 at 10:46, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:The attached version largely fixes with the performance degradation
that Tomas Vondra reported, replacing an O(N^2) piece of code in
slot_getmissingattrs() by one that is O(N).I've looked over this patch and had a play around with it.
Just a couple of things that I noticed while reading:
[ various typos ]
I'll try to spend some time reviewing the code in detail soon.
Thanks. I'll fix the typos in the next version of the patch, hopefully
after your review.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 19 February 2018 at 13:48, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Sun, Feb 18, 2018 at 2:52 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:I'll try to spend some time reviewing the code in detail soon.
Thanks. I'll fix the typos in the next version of the patch, hopefully
after your review.
Here's a more complete review... I reserve the right to be wrong about
a few things and be nitpicky on a few more...
3. All doc changes are indented 1 space too much. Many lines have also
been broken needlessly before 80 chars
4. Surplus "it" at end of line.
+ * Return the missing value of an attribute, or NULL if it
+ * there is no missing value.
5. "sttributes" -> "attributes"
+ /* initialize all the missing sttributes to null */
6. Surplus space before >=
+ if (missattn >= startAttNum && !attrmiss[i].ammissingNull)
7. Why bother with the else here?
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ }
+ else
+ {
+ missingnum = -1;
+ attrmiss = NULL;
+ }
You may as well just do:
+ if (!slot->tts_tupleDescriptor->constr)
+ return;
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
8. -1; should be - 1;
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing -1;
9. Surplus braces can be removed:
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
10. You've made a behavioral change in the following code:
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
Previously this just used to initialise up to attnum, but
slot_getmissingattrs does not know about attnum and always seems to
fill the entire slot with all atts in the TupleDesc
This may cause some performance regression when in cases where only
part of the tuple needs to be deformed. I think Tomas' benchmarks used
the final column in the table, so likely wouldn't have notice this,
although it may have if he'd aggregated one of the middle columns.
Once that's fixed, you could ditch the Min() call in the following code:
attno = Min(attno, attnum);
slot_deform_tuple(slot, attno);
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
* rest as NULLs or missing values
*/
if (attno < attnum)
{
slot_getmissingattrs(slot, attno);
}
And change it to something more like:
/*
* If tuple doesn't have all the atts indicated by tupleDesc, deform as
* much as possible, then fill the remaining required atts with nulls or
* missing values.
*/
if (attno < attnum)
{
slot_deform_tuple(slot, attno);
slot_getmissingattrs(slot, attno, attnum); // <-- (includes new parameter)
}
else
slot_deform_tuple(slot, attnum);
Which should result in a small performance increase due to not having
to compare attno < attnum twice. Although, perhaps the average compile
may optimise this anyway. I've not checked.
11. Is there a reason why the following code in getmissingattr() can't
be made into a bsearch?
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
As far as I can see, the attrmiss is sorted by amnum. But maybe I just
failed to find a case where it's not.
I'd imagine doing this would further improve Tomas' benchmark score
for the patched version, since he was performing a sum() on the final
column.
Although, If done, I'm actually holding some regard to the fact that
users may one day complain that their query became slower after a
table rewrite :-)
Update: I've stumbled upon the following code:
+ /*
+ * We have a dependency on the attrdef array being filled in in
+ * ascending attnum order. This should be guaranteed by the index
+ * driving the scan. But we want to be double sure
+ */
+ if (!(attp->attnum > attnum))
+ elog(ERROR, "attribute numbers not ascending");
and the code below this seems to also assemble the missing values in
attnum order.
While I'm here, I'd rather see this if test written as: if
(attp->attnum <= attnum)
Since the attnum order in the missing values appears to be well
defined in ascending order, you can also simplify the comparison logic
in equalTupleDescs(). The inner-most nested loop shouldn't be
required.
12. Additional braces not required in:
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ {
+ return false;
+ }
+ else
+ {
+ return true;
+ }
13. Also, just on study of the following condition:
+ if (tupleDesc &&
+ attnum <= tupleDesc->natts &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
I think that instead of testing "attnum <= tupleDesc->natts", you
should remove that condition an just add;
Assert(!tupleDesc || attnum <= tupleDesc->natts);
to the start of the function. No code should ever pass an attnum
greater than the tupleDesc's natts, so there's probably not much point
in checking that now if we didn't before.
14. In expand_tuple, the following can be declared in inner scope:
+ HeapTupleHeader targetTHeader;
(It would be nice if expand_tuple was a bit easier on the eye, but I
have no suggestions to improve it right now)
15. Why not: slot->tts_values[missattnum] = (Datum) 0;
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
There are a few other places you've done this. I'm unsure if there's
any sort of standard to follow.
16. Additional braces not required in:
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
Likewise in:
- for (; attnum < tdesc_natts; attnum++)
+ if (attnum < tdesc_natts)
{
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
+ slot_getmissingattrs(slot, attnum);
}
+
and:
- for (; attno < attnum; attno++)
+ if (attno < attnum)
{
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
+ slot_getmissingattrs(slot, attno);
}
maybe just fix this in all places your patch is touching.
17. Unrelated change:
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index ed90a02303..3b0be39101 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -416,7 +416,6 @@ sub morph_row_for_pgattr
}
elsif ($priornotnull)
{
-
# attnotnull will automatically be set if the type is
# fixed-width and prior columns are all NOT NULL ---
# compare DefineAttr in bootstrap.c. oidvector and
18. Can you explain what you mean by "doomed" here?
+ * If we add a column that is not null and there is no missing value
+ * (i.e. the missing value is NULL) then this ADD COLUMN is doomed.
+ * Unless the table is empty...
19. A few lines below are belong 80 chars:
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
Likewise in:
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
slot->tts_tupleDescriptor->natts)
and:
+ if (att_tup->atthasdef && rd_att->constr && rd_att->constr->num_defval > 0)
20. "missin" -> "missing", "Fill" -> "fill"
+ * If the column has a default or missin value Fill it into the
+ * attrdef array
21. Spaces instead of tabs. Maybe just pgindent the whole patch?
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
22. Missing <tab><space> before " 24"
+#define Anum_pg_attribute_attmissingval 24
Also, missing <tab> before " 15"
+#define Anum_pg_attribute_atthasmissing 15
23. Looks like you'll need to find a new home for the test, since the
comment claims we want no more than 19. You've added the 20th.
# NB: temp.sql does a reconnect which transiently uses 2 connections,
# so keep this parallel group to at most 19 tests
# ----------
-test: plancache limit plpgsql copy2 temp domain rangefuncs prepare
without_oid conversion truncate alter_table sequence polymorphism
rowtypes returning largeobject with xml
+test: plancache limit plpgsql copy2 temp domain rangefuncs prepare
without_oid conversion truncate alter_table sequence polymorphism
rowtypes returning largeobject with xml fast_default
I'll await the next version before looking again.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
http://www.publix.com/myaccount/verify?validationCode=889fd4cb-6dbb-4f93-9d4f-c701410cd8a2
On Mon, Feb 19, 2018 at 1:18 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:
On 19 February 2018 at 13:48, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:On Sun, Feb 18, 2018 at 2:52 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:I'll try to spend some time reviewing the code in detail soon.
Thanks. I'll fix the typos in the next version of the patch, hopefully
after your review.Here's a more complete review... I reserve the right to be wrong about
a few things and be nitpicky on a few more...
I've dealt with most of these issues. The only change of substance is
the suggested change in slot_getmissingattrs().
Once that's fixed, you could ditch the Min() call in the following code:
attno = Min(attno, attnum);
slot_deform_tuple(slot, attno);
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
* rest as NULLs or missing values
*/
if (attno < attnum)
{
slot_getmissingattrs(slot, attno);
}And change it to something more like:
/*
* If tuple doesn't have all the atts indicated by tupleDesc, deform as
* much as possible, then fill the remaining required atts with nulls or
* missing values.
*/
if (attno < attnum)
{
slot_deform_tuple(slot, attno);
slot_getmissingattrs(slot, attno, attnum); // <-- (includes new parameter)
}
else
slot_deform_tuple(slot, attnum);Which should result in a small performance increase due to not having
to compare attno < attnum twice. Although, perhaps the average compile
may optimise this anyway. I've not checked.
Maybe it would make a difference, but it seems unlikely to be significant.
11. Is there a reason why the following code in getmissingattr() can't
be made into a bsearch?
[...]
As far as I can see, the attrmiss is sorted by amnum. But maybe I just
failed to find a case where it's not.I'd imagine doing this would further improve Tomas' benchmark score
for the patched version, since he was performing a sum() on the final
column.Although, If done, I'm actually holding some regard to the fact that
users may one day complain that their query became slower after a
table rewrite :-)Update: I've stumbled upon the following code:
+ /* + * We have a dependency on the attrdef array being filled in in + * ascending attnum order. This should be guaranteed by the index + * driving the scan. But we want to be double sure + */ + if (!(attp->attnum > attnum)) + elog(ERROR, "attribute numbers not ascending");and the code below this seems to also assemble the missing values in
attnum order.While I'm here, I'd rather see this if test written as: if
(attp->attnum <= attnum)Since the attnum order in the missing values appears to be well
defined in ascending order, you can also simplify the comparison logic
in equalTupleDescs(). The inner-most nested loop shouldn't be
required.
Well, this comment in the existing code in equalTupleDescs() suggests
that in fact we can't rely on the attrdef items being in attnum order:
/*
* We can't assume that the items are always read from the system
* catalogs in the same order; so use the adnum field to identify
* the matching item to compare.
*/
And in any case the code out of date since the code no longer ties
missing values to default values. So I've removed the comment and the
error check.
There are probably a few optimisations we can make if we can assume
that the missing values are in the Tuple Descriptor are in attnum
order. But it's unclear to me why they should be and the attrdef
entries not.
15. Why not: slot->tts_values[missattnum] = (Datum) 0;
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
There are a few other places you've done this. I'm unsure if there's
any sort of standard to follow.
This is a pretty widely used idiom in the source.
I'll await the next version before looking again.
Thanks
attached.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v11.patchapplication/octet-stream; name=fast_default-v11.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..1914f7605e 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>bytea</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a binary representation of the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5be56d4b28..bb70bddd61 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1179,24 +1179,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..1f39a72eb1 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -76,6 +76,92 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ int missingnum;
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->num_missing > 0);
+
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Look through the tupledesc's attribute missing values
+ * for the one that corresponds to this attribute.
+ */
+ for (missingnum = tupleDesc->constr->num_missing - 1;
+ missingnum >= 0; missingnum--)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ {
+ *isnull = true;
+ return (Datum) 0;
+ }
+ else
+ {
+ *isnull = false;
+ return attrmiss[missingnum].ammissing;
+ }
+ }
+ }
+ Assert(false); /* should not be able to get here */
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss;
+ int missingnum;
+ int missattnum;
+ int i;
+
+ /* initialize all the missing attributes to null */
+ for (missattnum = lastAttNum - 1; missattnum >= startAttNum; missattnum--)
+ {
+ slot->tts_values[missattnum] = PointerGetDatum(NULL);
+ slot->tts_isnull[missattnum] = true;
+ }
+
+ /* now replace any we have a non-null missing value for */
+ if (slot->tts_tupleDescriptor->constr)
+ {
+ missingnum = slot->tts_tupleDescriptor->constr->num_missing - 1;
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+ for (i = missingnum; i >= 0; i--)
+ {
+ int missattn = attrmiss[i].amnum - 1;
+ if (missattn >= startAttNum && missattn < lastAttNum &&
+ !attrmiss[i].ammissingNull)
+ {
+ slot->tts_values[missattn] = attrmiss[i].ammissing;
+ slot->tts_isnull[missattn] = false;
+ }
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +218,131 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Fill in one attribute, either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ Assert(att);
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +383,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +408,18 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ return !(tupleDesc &&
+ TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing);
+ }
if (attnum > 0)
{
@@ -649,6 +772,293 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple with missing values, or NULLs. The source tuple must have
+ * less attributes than the required number. Only one of targetHeapTuple
+ * and targetMinimalTuple may be supplied. The other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int missingnum;
+ int firstmissingnum = 0;
+ int num_missing = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceIndicatorLen;
+ int targetIndicatorLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->num_missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ num_missing = tupleDesc->constr->num_missing;
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = 0;
+ firstmissingnum < num_missing
+ && attrmiss[firstmissingnum].amnum <= sourceNatts;
+ firstmissingnum++)
+ { /* empty */
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= num_missing)
+ {
+ hasNulls = true;
+ }
+
+ /*
+ * Now walk the missing attributes one by one. If there is a
+ * missing value make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = sourceNatts + 1;
+ attnum <= natts;
+ attnum++)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+
+ /*
+ * If there is a missing value for this attribute calculate
+ * the required space if it's not NULL.
+ */
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ if (attrmiss[missingnum].ammissingNull)
+ hasNulls = true;
+ else
+ {
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[missingnum].ammissing);
+ }
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ /* there is no missing value for this attribute */
+ hasNulls = true;
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetIndicatorLen = BITMAPLEN(natts);
+ len += targetIndicatorLen;
+ }
+ else
+ targetIndicatorLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetIndicatorLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetIndicatorLen > 0)
+ {
+ if (sourceIndicatorLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceIndicatorLen);
+ nullBits += sourceIndicatorLen - 1;
+ }
+ else
+ {
+ sourceIndicatorLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceIndicatorLen);
+
+ nullBits += sourceIndicatorLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ /* If there are no missing values there is nothing more to do */
+ if (firstmissingnum < num_missing)
+ {
+ targetData += sourceDataLen;
+ missingnum = firstmissingnum;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts + 1; attnum <= natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum - 1);
+
+ for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++)
+ {
+ if (attrmiss[missingnum].amnum == attnum)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[missingnum].ammissing,
+ attrmiss[missingnum].ammissingNull);
+ break;
+ }
+ }
+ if (missingnum > num_missing)
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+ } /* end if there is no missing value */
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1422,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as null, or a missing value if there is one.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1599,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1669,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1714,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1742,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1765,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..85383592ef 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -27,6 +27,7 @@
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +177,24 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if((cpy->num_missing = constr->num_missing) > 0)
+ {
+ cpy->missing = (AttrMissing *) palloc(cpy->num_missing * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, cpy->num_missing * sizeof(AttrMissing));
+ for (i = cpy->num_missing - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissing)
+ {
+ int attnum = constr->missing[i].amnum - 1;
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,20 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->num_missing > 0)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ Assert(attrmiss != NULL);
+
+ for (i = tupdesc->constr->num_missing - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissing
+ && !TupleDescAttr(tupdesc, attrmiss[i].amnum - 1)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +502,35 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ n = constr1->num_missing;
+ if (n != (int) constr2->num_missing)
+ return false;
+ for (i = 0; i < n; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing;
+
+ /* Similar logic to that used for defaults */
+ for (j = 0; j < n; missval2++, j++)
+ {
+ if (missval1->amnum == missval2->amnum)
+ break;
+ }
+ if (j >= n)
+ return false;
+ if (missval1->ammissingNull != missval2->ammissingNull)
+ return false;
+ if (missval1->ammissing && missval2->ammissing)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, missval1->amnum - 1);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ else if (missval1->ammissing || missval2->ammissing)
+ return false;
+ }
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +646,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,7 +860,9 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
+ constr->num_missing = 0;
constr->check = NULL;
constr->num_check = 0;
desc->constr = constr;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..35871eadab 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ char *missingBuf = NULL;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ exprState = ExecInitExpr(expr2, NULL);
+ estate = CreateExecutorState();
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ missingval = PointerGetDatum(NULL);
+ }
+ else if (defAttStruct->attbyval)
+ {
+ missingBuf = palloc(VARHDRSZ + sizeof(Datum));
+ memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum));
+ SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum));
+ missingval = PointerGetDatum(missingBuf);
+ }
+ else if (defAttStruct->attlen >= 0)
+ {
+ missingBuf = palloc(VARHDRSZ + defAttStruct->attlen);
+ memcpy(VARDATA(missingBuf), DatumGetPointer(missingval),
+ defAttStruct->attlen);
+ SET_VARSIZE(missingBuf,
+ VARHDRSZ + defAttStruct->attlen);
+ missingval = PointerGetDatum(missingBuf);
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2098,17 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (missingBuf)
+ {
+ pfree(missingBuf);
+ missingBuf = NULL;
+ }
+
return attrdefOid;
}
@@ -2180,7 +2261,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 564f2069cf..48bac61991 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1668,7 +1669,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..aa654067ba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index a027b19744..7228aa0d49 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2275,7 +2275,7 @@ ExecuteCallStmt(ParseState *pstate, CallStmt *stmt, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 3e48a58dcb..4f2bbe12b4 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index db6c8ff00e..843c0ff55d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4681,7 +4682,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4784,7 +4785,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5403,6 +5404,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5446,6 +5448,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5456,6 +5459,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5501,6 +5513,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5536,16 +5551,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6021,6 +6041,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8107,8 +8128,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9514,7 +9535,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 771b7e3945..e1c1fb741a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2483,7 +2483,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 91ba939bdc..aa5c7cbe0e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf93..025d3e379c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a8ae37ebc8..7012a2e8b3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..361bde4261 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index ba9fab4582..1cd5835686 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1230,7 +1230,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1396,7 +1396,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1629,7 +1629,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..7a34ef8507 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int nmissing = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,8 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +560,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +568,73 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ /* Do we have a missing value? */
+ missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ &missingNull);
+ attrmiss[nmissing].amnum = attnum;
+ if (missingNull)
+ {
+ /* No, then the store a NULL */
+ attrmiss[nmissing].ammissing = PointerGetDatum(NULL);
+ attrmiss[nmissing].ammissingNull = true;
+ }
+ else if (attp->attbyval)
+ {
+ /*
+ * Yes, and it's of the by-value kind. Copy in the Datum
+ */
+ memcpy(&attrmiss[nmissing].ammissing,
+ VARDATA_ANY(missingval), sizeof(Datum));
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else if (attp->attlen > 0)
+ {
+ /*
+ * Yes, and it's fixed length. Copy it out and have the Datum
+ * point to it.
+ */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = PointerGetDatum(palloc(attp->attlen));
+ memcpy(DatumGetPointer(attrmiss[nmissing].ammissing),
+ VARDATA_ANY(missingval), attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ attrmiss[nmissing].ammissingNull = false;
+ }
+ else
+ {
+ /* Yes, variable length */
+ MemoryContext oldcxt;
+
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[nmissing].ammissing = datumCopy(missingval,
+ attp->attbyval,
+ attp->attlen);
+ attrmiss[nmissing].ammissingNull = false;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ nmissing++;
+ }
need--;
if (need == 0)
break;
@@ -605,7 +675,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ nmissing > 0 || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +691,19 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+
+ if (nmissing > 0)
+ {
+ if (nmissing < relation->rd_rel->relnatts)
+ constr->missing = (AttrMissing *)
+ repalloc(attrmiss, nmissing * sizeof(AttrMissing));
+ else
+ constr->missing = attrmiss;
+ }
+ constr->num_missing = nmissing;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4142,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4480,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4775,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4838,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..ed30f98e33 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,6 +14,7 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -25,6 +26,13 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing
+{
+ AttrNumber amnum;
+ bool ammissingNull; /* true if missing value is NULL */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,8 +46,10 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ AttrMissing *missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
+ uint16 num_missing;
bool has_not_null;
} TupleConstr;
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..7ee68a6172 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /* Missing value for added columns */
+ bytea attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +197,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +212,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..9e74068a0a
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..c4bf9d61e4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing
+test: identity partition_join partition_prune reloptions hash_part indexing fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..6a3f9b367e
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
Hi,
On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote:
Anyway, I consider the performance to be OK. But perhaps Andres could
comment on this too, as he requested the benchmarks.
My performance concerns were less about CREATE TABLE related things than
about analytics workloads or such, where deforming is the primary
bottleneck. I think it should be ok, but doing a before/after tpc-h of
scale 5-10 or so wouldn't be a bad thing to verify.
Greetings,
Andres Freund
On Tue, Feb 20, 2018 at 5:03 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
http://www.publix.com/myaccount/verify?validationCode=889fd4cb-6dbb-4f93-9d4f-c701410cd8a2
Wow, my c&p was working overtime. Good thing this won't do anyone any good.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Thanks for making those updates
.
On 20 February 2018 at 19:33, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Mon, Feb 19, 2018 at 1:18 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:Since the attnum order in the missing values appears to be well
defined in ascending order, you can also simplify the comparison logic
in equalTupleDescs(). The inner-most nested loop shouldn't be
required.Well, this comment in the existing code in equalTupleDescs() suggests
that in fact we can't rely on the attrdef items being in attnum order:/*
* We can't assume that the items are always read from the system
* catalogs in the same order; so use the adnum field to identify
* the matching item to compare.
*/
On closer look, perhaps the reason that comment claims the order is
not well defined is that the use of the index depends on the value of
criticalRelcachesBuilt in the following:
pg_attribute_scan = systable_beginscan(pg_attribute_desc,
AttributeRelidNumIndexId,
criticalRelcachesBuilt,
NULL,
2, skey);
Otherwise, I see no reason for them to be out of order, as if I grep
for instances of "->missing = " I only see the place where the missing
attr details are set in RelationBuildTupleDesc() and one other place
in CreateTupleDescCopyConstr(), where it's simply just copied via
memcpy().
Nevertheless, it would be interesting to see how much a bsearch in
getmissingattr would help Tomas' case. Though, perhaps you're happy
enough with the performance already.
I'll make another pass over the updated code soon to see if I can see
anything else.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 20 February 2018 at 23:10, David Rowley <david.rowley@2ndquadrant.com> wrote:
Nevertheless, it would be interesting to see how much a bsearch in
getmissingattr would help Tomas' case. Though, perhaps you're happy
enough with the performance already.
I thought I'd give this a quick test using the attached (incomplete) patch.
My findings made me realise that Tomas' case actually exploited a best
case for the patch.
Tomas' query.sql is performing a sum(c1000), which is the final column
in the table. No doubt he's done this since it's the typical thing to
do to get a worst case for tuple deforming, but he might not have
realised it was the best case for your patch due to the way you're
performing the for loop in getmissingattr starting with the final
missing value first.
I noticed this when I tested the mockup bsearch code. I was surprised
to see that it actually slowed performance a little with the
sum(c1000) case.
The entire results for those using:
pgbench -n -T 60 -j 1 -c 1 -f query.sql postgres
is:
Using: select sum(c1000) from t;
v11 + bsearch + create.sql: tps = 1479.267518
v11 + bsearch + create-alter.sql: tps = 1366.885968
v11 + create.sql: tps = 1544.378091
v11 + create-alter.sql: tps = 1416.608320
(both the create.sql results should be about the same here since
they're not really exercising any new code.)
Notice that the bsearch version is slightly slower, 1366 tps vs 1416.
If I use sum(c10) instead, I get.
Using: select sum(c10) from t;
v11 + bsearch + create-alter.sql: tps = 1445.940061
v11 + bsearch + create.sql: tps = 3320.102543
v11 + create.sql: tps = 3330.131437
v11 + create-alter.sql: tps = 1398.635251
The bsearch here does help recover some performance, but it's a small
improvement on what is quite a big drop from the create.sql case.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
bsearch_getmissingattr.patchapplication/octet-stream; name=bsearch_getmissingattr.patchDownload
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 1f39a72..03d47b6 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -83,7 +83,6 @@ static Datum
getmissingattr(TupleDesc tupleDesc,
int attnum, bool *isnull)
{
- int missingnum;
Form_pg_attribute att;
AttrMissing *attrmiss;
@@ -94,21 +93,33 @@ getmissingattr(TupleDesc tupleDesc,
if (att->atthasmissing)
{
+ int lo,
+ hi,
+ mid;
+
Assert(tupleDesc->constr);
Assert(tupleDesc->constr->num_missing > 0);
attrmiss = tupleDesc->constr->missing;
+ lo = -1;
+ hi = tupleDesc->constr->num_missing - 1;
+
/*
- * Look through the tupledesc's attribute missing values
- * for the one that corresponds to this attribute.
+ * Binary search the tupledesc's attribute missing values for the one
+ * that corresponds to this attribute.
*/
- for (missingnum = tupleDesc->constr->num_missing - 1;
- missingnum >= 0; missingnum--)
+ while (lo < hi)
{
- if (attrmiss[missingnum].amnum == attnum)
+ mid = (lo + hi + 1) / 2;
+
+ if (attrmiss[mid].amnum < attnum)
+ lo = mid;
+ else if (attrmiss[mid].amnum > attnum)
+ hi = mid - 1;
+ else
{
- if (attrmiss[missingnum].ammissingNull)
+ if (attrmiss[mid].ammissingNull)
{
*isnull = true;
return (Datum) 0;
@@ -116,10 +127,11 @@ getmissingattr(TupleDesc tupleDesc,
else
{
*isnull = false;
- return attrmiss[missingnum].ammissing;
+ return attrmiss[mid].ammissing;
}
}
}
+
Assert(false); /* should not be able to get here */
}
On 20/02/18 07:42, Andres Freund wrote:
Hi,
On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote:
Anyway, I consider the performance to be OK. But perhaps Andres could
comment on this too, as he requested the benchmarks.My performance concerns were less about CREATE TABLE related things than
about analytics workloads or such, where deforming is the primary
bottleneck. I think it should be ok, but doing a before/after tpc-h of
scale 5-10 or so wouldn't be a bad thing to verify.
The test Tomas is doing is analytical query, it's running sum on the new
fast default column.
He uses create and create-alter names as comparison between when the
table was created with the columns and when the columns were added using
fast default.
--
Petr Jelinek http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On February 20, 2018 5:03:58 AM PST, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
On 20/02/18 07:42, Andres Freund wrote:
Hi,
On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote:
Anyway, I consider the performance to be OK. But perhaps Andres
could
comment on this too, as he requested the benchmarks.
My performance concerns were less about CREATE TABLE related things
than
about analytics workloads or such, where deforming is the primary
bottleneck. I think it should be ok, but doing a before/after tpc-hof
scale 5-10 or so wouldn't be a bad thing to verify.
The test Tomas is doing is analytical query, it's running sum on the
new
fast default column.He uses create and create-alter names as comparison between when the
table was created with the columns and when the columns were added
using
fast default.
It's still a fairly simplistic test case. Running some queries with reasonably well known characteristics seems like a good idea regardless. It's not like a scale 5 run takes that long.
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
... btw, I've not read this patch in any detail, but the recent thread
about toast tables for system catalogs prompts me to wonder what happens
if a "fast default" value is large enough to require out-of-line toasting.
I can easily think of problems that will ensue if we try to support that
case, because right now the toast mechanisms assume that OOL toasted
values can only be referenced from the associated table.
regards, tom lane
Hi,
On 2018-02-20 13:07:30 -0500, Tom Lane wrote:
... btw, I've not read this patch in any detail, but the recent thread
about toast tables for system catalogs prompts me to wonder what happens
if a "fast default" value is large enough to require out-of-line toasting.
Hm, interesting.
I can easily think of problems that will ensue if we try to support that
case, because right now the toast mechanisms assume that OOL toasted
values can only be referenced from the associated table.
What problem are you seeing with treating the toasted value to be from
pg_attribute? I'm only drinking my first coffee just now, so I might be
missing something...
Now we certainly would need to make sure that the corresponding
pg_attribute row containing the default value doesn't go away too early,
but that shouldn't be much of a problem given that we never remove
them. I wondered for a second if there's problematic cases where the
default value is referenced by an index, and then the default-adding
transaction rolls back. But I can't construct anything realistically
problematic.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2018-02-20 13:07:30 -0500, Tom Lane wrote:
I can easily think of problems that will ensue if we try to support that
case, because right now the toast mechanisms assume that OOL toasted
values can only be referenced from the associated table.
What problem are you seeing with treating the toasted value to be from
pg_attribute? I'm only drinking my first coffee just now, so I might be
missing something...
Locking/vacuuming is exactly what I'm worried about. Right now, a
transaction cannot reference a toast value without holding at least
AccessShare on the parent table. That would not be true of OOL fast
defaults that are living in pg_attribute's toast table (if it had one).
If you think this isn't potentially problematic, see the problems that
forced us into hacks like 08e261cbc.
I guess the fix equivalent to 08e261cbc would be to require any toasted
fast-default value to be detoasted into line whenever it's copied into
a tupledesc, rather than possibly being fetched later.
Now we certainly would need to make sure that the corresponding
pg_attribute row containing the default value doesn't go away too early,
but that shouldn't be much of a problem given that we never remove
them. I wondered for a second if there's problematic cases where the
default value is referenced by an index, and then the default-adding
transaction rolls back. But I can't construct anything realistically
problematic.
Oooh ... but no, because we don't allow toasted values to be copied into
indexes, for (I think) exactly this reason. See index_form_tuple.
regards, tom lane
Hi,
On 2018-02-20 13:50:54 -0500, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On 2018-02-20 13:07:30 -0500, Tom Lane wrote:
I can easily think of problems that will ensue if we try to support that
case, because right now the toast mechanisms assume that OOL toasted
values can only be referenced from the associated table.What problem are you seeing with treating the toasted value to be from
pg_attribute? I'm only drinking my first coffee just now, so I might be
missing something...Locking/vacuuming is exactly what I'm worried about. Right now, a
transaction cannot reference a toast value without holding at least
AccessShare on the parent table.
Is that actually reliably true? Doesn't e.g. use a variable from a
rolled-back subtransaction in plpgsql circumvent this already? There
are a few bugs around this though, so that's not really a convincing
argument of there not being any danger :/
That would not be true of OOL fast defaults that are living in
pg_attribute's toast table (if it had one). If you think this isn't
potentially problematic, see the problems that forced us into hacks
like 08e261cbc.
I don't think the equivalent problem quite applies here, the OOL default
should afaics be immutable. But yea, it's a concern.
I guess the fix equivalent to 08e261cbc would be to require any toasted
fast-default value to be detoasted into line whenever it's copied into
a tupledesc, rather than possibly being fetched later.
Yea, thats probably a good idea. Correctness aside, it's probably better
to detoast once rather than do so for every single row for performance
reasons (detoasting the same row over and over in multiple backends
would lead to quite the contention on used buffer headers and relation
locks).
Now we certainly would need to make sure that the corresponding
pg_attribute row containing the default value doesn't go away too early,
but that shouldn't be much of a problem given that we never remove
them. I wondered for a second if there's problematic cases where the
default value is referenced by an index, and then the default-adding
transaction rolls back. But I can't construct anything realistically
problematic.Oooh ... but no, because we don't allow toasted values to be copied into
indexes, for (I think) exactly this reason. See index_form_tuple.
I don't think that'd necessarily provide full protection - what about a
index recheck during a lossy bitmap index scan for example? But given
that any such index creation would also be rolled back I'm not too
concerned.
Greetings,
Andres Freund
On 02/20/2018 06:43 PM, Andres Freund wrote:
On February 20, 2018 5:03:58 AM PST, Petr Jelinek <petr.jelinek@2ndquadrant.com> wrote:
On 20/02/18 07:42, Andres Freund wrote:
Hi,
On 2018-02-17 00:23:40 +0100, Tomas Vondra wrote:
Anyway, I consider the performance to be OK. But perhaps Andres
could
comment on this too, as he requested the benchmarks.
My performance concerns were less about CREATE TABLE related things
than
about analytics workloads or such, where deforming is the primary
bottleneck. I think it should be ok, but doing a before/after tpc-hof
scale 5-10 or so wouldn't be a bad thing to verify.
The test Tomas is doing is analytical query, it's running sum on the
new
fast default column.He uses create and create-alter names as comparison between when
the table was created with the columns and when the columns were
added using fast default.It's still a fairly simplistic test case. Running some queries with
reasonably well known characteristics seems like a good idea
regardless. It's not like a scale 5 run takes that long.
I can do that, although I don't really expect any surprises there.
The simple tests I did were supposed to test two corner cases - best
case, when there are no columns with "fast default", and worst case when
all columns (and tuples) have "fast default". Per David's findings, the
second test was not perfectly right, but that's a separate issue.
The question is how should the schema for TPC-H look like. Because if
you just do the usual test without any ALTER TABLE ADD COLUMN, then you
really won't get any difference at all. The fast default stuff will be
completely "inactive".
So, we need to tweak the TPC-H schema somehow ... We need (a) to add
some new columns with default values, (b) tweak some of the queries to
use those new columns. Suggestions?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote:
The question is how should the schema for TPC-H look like. Because if
you just do the usual test without any ALTER TABLE ADD COLUMN, then you
really won't get any difference at all. The fast default stuff will be
completely "inactive".
Well, there'll still be changed behaviour due to the changed sizes of
structures and new out of line function calls. The deforming code is
quite sensitive to such changes.
I think just verifying that there's no meaningful effect with/without
the patch is good enough. As soon as there's actual new columns that
take advantage of the new functionality I think some degradation is fair
game, you got some benefit for it.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote:
The question is how should the schema for TPC-H look like. Because if
you just do the usual test without any ALTER TABLE ADD COLUMN, then you
really won't get any difference at all. The fast default stuff will be
completely "inactive".
I think just verifying that there's no meaningful effect with/without
the patch is good enough. As soon as there's actual new columns that
take advantage of the new functionality I think some degradation is fair
game, you got some benefit for it.
Yeah, I think what we want to know is how much penalty people are paying
for the feature to exist even though they aren't using it. That seems
to break down into two cases:
(1) use of a table that's never had any added columns;
(2) use of a table that has had columns added, but they just default
to null the same as before. Is there more relcache setup time anyway?
Is deforming slower even if there are no fast defaults?
Case (1) could be answered by a straight TPC-H test with/without patch.
Case (2) seems a bit harder, since presumably you want to measure accesses
to tuples that are "short" compared to the current table tupdesc, but for
which the extra columns are still default-null.
regards, tom lane
On 02/20/2018 09:14 PM, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On 2018-02-20 20:57:36 +0100, Tomas Vondra wrote:
The question is how should the schema for TPC-H look like. Because if
you just do the usual test without any ALTER TABLE ADD COLUMN, then you
really won't get any difference at all. The fast default stuff will be
completely "inactive".I think just verifying that there's no meaningful effect with/without
the patch is good enough. As soon as there's actual new columns that
take advantage of the new functionality I think some degradation is fair
game, you got some benefit for it.Yeah, I think what we want to know is how much penalty people are paying
for the feature to exist even though they aren't using it. That seems
to break down into two cases:(1) use of a table that's never had any added columns;
(2) use of a table that has had columns added, but they just default
to null the same as before. Is there more relcache setup time anyway?
Is deforming slower even if there are no fast defaults?Case (1) could be answered by a straight TPC-H test with/without patch.
I don't quite understand why would this case need the TPC-H tests, or
why would TPC-H give us more than the very focused tests we've already
done. The first test was testing a fairly short query where any such
additional overhead would be much more obvious, compared to the TPC-H
queries that usually do a lot of other expensive stuff.
I'm fine with doing the tests, of course.
Case (2) seems a bit harder, since presumably you want to measure
accesses to tuples that are "short" compared to the current table
tupdesc, but for which the extra columns are still default-null.
Yeah. I think we can add a column or two to the "fact" tables
(particularly lineitem), and then tweak the queries to also compute
aggregates on this new column.
Unless someone has better ideas, I'll do such tests once the machine
completes the stuff that's running now.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On 2018-02-20 21:28:40 +0100, Tomas Vondra wrote:
I don't quite understand why would this case need the TPC-H tests, or
why would TPC-H give us more than the very focused tests we've already
done.
Because a more complex query shows the cost of changing cache access
costs better than a trivial query. Simplistic queries will often
e.g. not show cost of additional branch predictor usage, because the
branch history is large enough to fit the simple query. But once you go
to a more complex query, and that's not necessarily the case anymore.
The first test was testing a fairly short query where any such
additional overhead would be much more obvious, compared to the TPC-H
queries that usually do a lot of other expensive stuff.
Unfortunately such reasoning IME doesn't work well with cpu-bound stuff.
Greetings,
Andres Freund
On 02/20/2018 09:36 PM, Andres Freund wrote:
Hi,
On 2018-02-20 21:28:40 +0100, Tomas Vondra wrote:
I don't quite understand why would this case need the TPC-H tests, or
why would TPC-H give us more than the very focused tests we've already
done.Because a more complex query shows the cost of changing cache access
costs better than a trivial query. Simplistic queries will often
e.g. not show cost of additional branch predictor usage, because the
branch history is large enough to fit the simple query. But once you go
to a more complex query, and that's not necessarily the case anymore.The first test was testing a fairly short query where any such
additional overhead would be much more obvious, compared to the TPC-H
queries that usually do a lot of other expensive stuff.Unfortunately such reasoning IME doesn't work well with cpu-bound stuff.
OK, point taken. I'll do the tests and report the results.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 21 February 2018 at 00:38, David Rowley <david.rowley@2ndquadrant.com> wrote:
Using: select sum(c10) from t;
...
v11 + create.sql: tps = 3330.131437
v11 + create-alter.sql: tps = 1398.635251
It seems the difference between these two cases is down to
slot_getsomeattrs being asked to deform up to attnum 1000 for the
create-alter.sql case, and only up to attnum 10 for the create.sql
case. Both plans are using physical tlists per EXPLAIN VERBOSE. I've
not managed to narrow down the reason for the difference yet. Looks
like it might take a bit of time with a debugger to find the point
where the code paths of the two cases diverge.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
David Rowley <david.rowley@2ndquadrant.com> writes:
It seems the difference between these two cases is down to
slot_getsomeattrs being asked to deform up to attnum 1000 for the
create-alter.sql case, and only up to attnum 10 for the create.sql
case. Both plans are using physical tlists per EXPLAIN VERBOSE. I've
not managed to narrow down the reason for the difference yet.
There's logic in the execExpr compiler that detects the last attnum
we actually reference, if memory serves.
regards, tom lane
Hi,
On 2018-02-20 17:03:07 +1030, Andrew Dunstan wrote:
+ <row> + <entry><structfield>attmissingval</structfield></entry> + <entry><type>bytea</type></entry> + <entry></entry> + <entry> + This column has a binary representation of the value used when the + column is entirely missing from the row, as happens when the column is + added after the row is created. The value is only used when + <structname>atthasmissing</structname> is true. + </entry> + </row>
Hm. Why is this a bytea, and why is that a good idea? In pg_statistics,
which has to deal with something similar-ish, we deal with the ambiguity
of the storage by storing anyarrays, which then can display the contents
properly thanks to the embedded oid. Now it'd be a bit sad to store a
one element array in the table. But that seems better than storing a
bytea that's typeless and can't be viewed reasonably. The best solution
would be to create a variadic any type that can actually be stored, but
I see why you'd not want to go there necessarily.
But also, a bit higher level, is it a good idea to store this
information directly into pg_attribute? That table is already one of the
hotter system catalog tables, and very frequently the largest. If we
suddenly end up storing a lot of default values in there, possibly ones
that aren't ever actually used because all the default values are long
since actually stored in the table, we'll spend more time doing cache
lookups.
I'm somewhat tempted to say that the default data should be in a
separate catalog table. Or possibly use pg_attrdef.
+/* + * Return the missing value of an attribute, or NULL if there isn't one. + */ +static Datum +getmissingattr(TupleDesc tupleDesc, + int attnum, bool *isnull) +{ + int missingnum; + Form_pg_attribute att; + AttrMissing *attrmiss; + + Assert(attnum <= tupleDesc->natts); + Assert(attnum > 0); + + att = TupleDescAttr(tupleDesc, attnum - 1); + + if (att->atthasmissing) + { + Assert(tupleDesc->constr); + Assert(tupleDesc->constr->num_missing > 0); + + attrmiss = tupleDesc->constr->missing; + + /* + * Look through the tupledesc's attribute missing values + * for the one that corresponds to this attribute. + */ + for (missingnum = tupleDesc->constr->num_missing - 1; + missingnum >= 0; missingnum--) + { + if (attrmiss[missingnum].amnum == attnum) + { + if (attrmiss[missingnum].ammissingNull) + { + *isnull = true; + return (Datum) 0; + } + else + { + *isnull = false; + return attrmiss[missingnum].ammissing; + } + } + }
I think requiring this is a bad idea. If, and only if, there's default
attributes with missing values we should build an array that can be
directly indexed by attribute number, accessing the missing value. What
you're doing here is essentially O(n^2) afaict, with n being the number
of columns missing values.
+/* + * Fill in missing values for a TupleTableSlot + */ +static void +slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum) +{ + AttrMissing *attrmiss; + int missingnum; + int missattnum; + int i; + + /* initialize all the missing attributes to null */ + for (missattnum = lastAttNum - 1; missattnum >= startAttNum; missattnum--) + { + slot->tts_values[missattnum] = PointerGetDatum(NULL); + slot->tts_isnull[missattnum] = true; + }
Any reason not to memset this in one (well two) go with memset? Also,
how come you're using PointerGetDatum()? Normally we just use (Datum) 0,
imo it's confusing to use PointerGetDatum(), as it implies things that
are quite possibly not the case.
+/* + * Fill in one attribute, either a data value or a bit in the null bitmask + */
Imo this isn't sufficient documentation to explain what this function
does. Even if you just add "per-attribute helper for heap_fill_tuple
and other routines building tuples" or such, I think it'd be clearer.
+static inline void +fill_val(Form_pg_attribute att, + bits8 **bit, + int *bitmask, + char **dataP, + uint16 *infomask, + Datum datum, + bool isnull) +{ + Size data_length; + char *data = *dataP; + + /* + * If we're building a null bitmap, set the appropriate bit for the + * current column value here. + */ + if (bit != NULL) + { + if (*bitmask != HIGHBIT) + *bitmask <<= 1; + else + { + *bit += 1; + **bit = 0x0; + *bitmask = 1; + } + + if (isnull) + { + *infomask |= HEAP_HASNULL; + return; + } + + **bit |= *bitmask; + } + + Assert(att);
Why isn't this asserted at the beginning?
@@ -293,10 +408,18 @@ heap_fill_tuple(TupleDesc tupleDesc, * ---------------- */ bool -heap_attisnull(HeapTuple tup, int attnum) +heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc) { + /* + * We allow a NULL tupledesc for relations not expected to have + * missing values, such as catalog relations and indexes. + */ + Assert(!tupleDesc || attnum <= tupleDesc->natts);
This seems fairly likely to hide bugs.
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data)) - return true; + { + return !(tupleDesc && + TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing);
Brrr.
+/* + * Expand a tuple with missing values, or NULLs. The source tuple must have + * less attributes than the required number. Only one of targetHeapTuple + * and targetMinimalTuple may be supplied. The other argument must be NULL. + */
I can't quite make head or tails of this comment "with missing values, or NULLs"?
+static void +expand_tuple(HeapTuple *targetHeapTuple, + MinimalTuple *targetMinimalTuple, + HeapTuple sourceTuple, + TupleDesc tupleDesc) +{ + AttrMissing *attrmiss = NULL; + int attnum; + int missingnum; + int firstmissingnum = 0; + int num_missing = 0; + bool hasNulls = HeapTupleHasNulls(sourceTuple); + HeapTupleHeader targetTHeader; + HeapTupleHeader sourceTHeader = sourceTuple->t_data; + int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader); + int natts = tupleDesc->natts; + int sourceIndicatorLen; + int targetIndicatorLen; + Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff; + Size targetDataLen; + Size len; + int hoff; + bits8 *nullBits = NULL; + int bitMask = 0; + char *targetData; + uint16 *infoMask; + + Assert((targetHeapTuple && !targetMinimalTuple) + || (!targetHeapTuple && targetMinimalTuple)); + + Assert(sourceNatts < natts); + + sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
Maybe I'm just grumpy right now (did response work on 2016 taxes just
now, brrrrr), but I don't find "indicatorLen" a descriptive term.
+ targetDataLen = sourceDataLen; + + if (tupleDesc->constr && + tupleDesc->constr->num_missing) + { + /* + * If there are missing values we want to put them into the tuple. + * Before that we have to compute the extra length for the values + * array and the variable length data. + */ + num_missing = tupleDesc->constr->num_missing; + attrmiss = tupleDesc->constr->missing;
You're accessing a hell of a lot of expensive stuff outside outside of
this if block. E.g. HeapTupleHeaderGetNatts specifically is the prime
source of cache misses in a lot of workloads. Now a smart compiler might
be able to optimize this away, but...
+ /* + * Find the first item in attrmiss for which we don't have a value in + * the source (i.e. where its amnum is > sourceNatts). We can ignore + * all the missing entries before that. + */ + for (firstmissingnum = 0; + firstmissingnum < num_missing + && attrmiss[firstmissingnum].amnum <= sourceNatts; + firstmissingnum++) + { /* empty */ + } + + /* + * If there are no more missing values everything else must be + * NULL + */ + if (firstmissingnum >= num_missing) + { + hasNulls = true; + } + + /* + * Now walk the missing attributes one by one. If there is a + * missing value make space for it. Otherwise, it's going to be NULL. + */ + for (attnum = sourceNatts + 1; + attnum <= natts; + attnum++) + { + Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1); + + for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++) + { + + /* + * If there is a missing value for this attribute calculate + * the required space if it's not NULL. + */ + if (attrmiss[missingnum].amnum == attnum) + { + if (attrmiss[missingnum].ammissingNull) + hasNulls = true; + else + { + targetDataLen = att_align_datum(targetDataLen, + att->attalign, + att->attlen, + attrmiss[missingnum].ammissing); + + targetDataLen = att_addlength_pointer(targetDataLen, + att->attlen, + attrmiss[missingnum].ammissing); + } + break; + } + } + if (missingnum > num_missing) + { + /* there is no missing value for this attribute */ + hasNulls = true; + } + } + } /* end if have missing values */ + else + { + /* + * If there are no missing values at all then NULLS must be allowed, + * since some of the attributes are known to be absent. + */ + hasNulls = true;
Why are we doing anything at all in this branch? ISTM we're reforming
the tuple for absolutely no gain here, just making it larger by
including additionall null bits?
+ /* + * Allocate and zero the space needed. Note that the tuple body and + * HeapTupleData management structure are allocated in one chunk. + */ + if (targetHeapTuple) + { + len += offsetof(HeapTupleHeaderData, t_bits); + hoff = len = MAXALIGN(len); /* align user data safely */ + len += targetDataLen;
So len isn't the length of the tuple, but the null bitmap length? That
seems, uh, non-obvious.
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len); + (*targetHeapTuple)->t_data + = targetTHeader + = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
Yuck.
+ (*targetHeapTuple)->t_len = len; + (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid; + ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
Seems like a lot of this code would be more readable by storing
*targetHeapTuple in a local var.
+ targetTHeader->t_infomask = sourceTHeader->t_infomask; + targetTHeader->t_hoff = hoff; + HeapTupleHeaderSetNatts(targetTHeader, natts); + HeapTupleHeaderSetDatumLength(targetTHeader, len); + HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid); + HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod); + /* We also make sure that t_ctid is invalid unless explicitly set */ + ItemPointerSetInvalid(&(targetTHeader->t_ctid));
This seems fairly expensive, why aren't we just memcpy'ing the old
header into the new one?
+ if (targetIndicatorLen > 0) + { + if (sourceIndicatorLen > 0) + { + /* if bitmap pre-existed copy in - all is set */
can't parse.
+ memcpy(nullBits, + ((char *) sourceTHeader) + + offsetof(HeapTupleHeaderData, t_bits), + sourceIndicatorLen); + nullBits += sourceIndicatorLen - 1; + } + else + { + sourceIndicatorLen = BITMAPLEN(sourceNatts); + /* Set NOT NULL for all existing attributes */ + memset(nullBits, 0xff, sourceIndicatorLen); + + nullBits += sourceIndicatorLen - 1; + + if (sourceNatts & 0x07) + { + /* build the mask (inverted!) */ + bitMask = 0xff << (sourceNatts & 0x07); + /* Voila */ + *nullBits = ~bitMask; + }
It's late, I'm tired. But how can this be correct? Also, this code is
massively under-commented.
+ +/* + * Fill in the missing values for an ordinary HeapTuple + */ +MinimalTuple +minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc) +{ + MinimalTuple minimalTuple; + + expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc); + return minimalTuple; +} + +/* + * Fill in the missing values for a minimal HeapTuple + */ +HeapTuple +heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc) +{ + HeapTuple heapTuple; + + expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc); + return heapTuple; +}
Comments appear to be swapped.
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1422,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,/* * If tuple doesn't have all the atts indicated by tupleDesc, read the - * rest as null + * rest as null, or a missing value if there is one. */
"a missing value" isn't there quite possibly multiple ones?
/* @@ -1192,8 +1599,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull) tup = tuple->t_data; if (attnum > HeapTupleHeaderGetNatts(tup)) { - *isnull = true; - return (Datum) 0; + return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull); }
This turned a ~4 instruction case into considerably more. I've recently
removed one of the major slot_getattr sources (hashagg comparisons), but
we still call this super frequently for hashagg's hash computations. I
suggest micro-benchmarking that case.
Oid StoreAttrDefault(Relation rel, AttrNumber attnum, - Node *expr, bool is_internal) + Node *expr, bool is_internal, bool add_column_mode) { char *adbin; char *adsrc; @@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, Relation attrrel; HeapTuple atttup; Form_pg_attribute attStruct; + Form_pg_attribute defAttStruct; Oid attrdefOid; ObjectAddress colobject, defobject; + ExprState *exprState; + Expr *expr2 = (Expr *) expr; + EState *estate = NULL; + ExprContext *econtext; + char *missingBuf = NULL; + Datum valuesAtt[Natts_pg_attribute]; + bool nullsAtt[Natts_pg_attribute]; + bool replacesAtt[Natts_pg_attribute]; + Datum missingval = (Datum) 0; + bool missingIsNull = true;/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);+ /* + * Compute the missing value + */ + expr2 = expression_planner(expr2); + + exprState = ExecInitExpr(expr2, NULL);
This should instead use ExecPrepareExpr()
+ if (add_column_mode) + { + missingval = ExecEvalExpr(exprState, econtext, + &missingIsNull); + + defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1); + + if (missingIsNull) + { + missingval = PointerGetDatum(NULL);
(Datum) 0;
+ } + else if (defAttStruct->attbyval) + { + missingBuf = palloc(VARHDRSZ + sizeof(Datum)); + memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum)); + SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum)); + missingval = PointerGetDatum(missingBuf); + } + else if (defAttStruct->attlen >= 0) + { + missingBuf = palloc(VARHDRSZ + defAttStruct->attlen); + memcpy(VARDATA(missingBuf), DatumGetPointer(missingval), + defAttStruct->attlen); + SET_VARSIZE(missingBuf, + VARHDRSZ + defAttStruct->attlen); + missingval = PointerGetDatum(missingBuf); + }
What if we get an extended datum back in the varlena case? Even if we
wouldn't change the storage from bytea to something else, you really
would need to force detoasting here.
+ } + /* * Make the pg_attrdef entry. */ @@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, attStruct = (Form_pg_attribute) GETSTRUCT(atttup); if (!attStruct->atthasdef) { - attStruct->atthasdef = true; + MemSet(valuesAtt, 0, sizeof(valuesAtt)); + MemSet(nullsAtt, false, sizeof(nullsAtt)); + MemSet(replacesAtt, false, sizeof(replacesAtt)); + valuesAtt[Anum_pg_attribute_atthasdef - 1] = true; + replacesAtt[Anum_pg_attribute_atthasdef - 1] = true; + if (add_column_mode) + { + valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true; + replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true; + valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval; + replacesAtt[Anum_pg_attribute_attmissingval - 1] = true; + nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull; + } + atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel), + valuesAtt, nullsAtt, replacesAtt); +
This seems weird. We've above formed a tuple, now we just modify it
again?
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal); + /* If the default is volatile we cannot use a missing value */ + if (contain_volatile_functions((Node *) expr)) + { + colDef->missingMode = false; + } + defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal, + colDef->missingMode);
Doesn't this have to be immutable rather than just stable? I mean just
storing the value of now() doesn't seem like it'd do the trick?
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot) * If we have a regular physical tuple then just return it. */ if (TTS_HAS_PHYSICAL_TUPLE(slot)) - return slot->tts_tuple; + { + if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < + slot->tts_tupleDescriptor->natts) + { + MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt); + + slot->tts_tuple = heap_expand_tuple(slot->tts_tuple, + slot->tts_tupleDescriptor); + slot->tts_shouldFree = true; + MemoryContextSwitchTo(oldContext); + return slot->tts_tuple; + } + else + { + return slot->tts_tuple; + } + }
I'm extremely dubious about this. HeapTupleHeaderGetNatts() commonly
causes cachemisses and therefore causes slowdowns when unnecessarily
done. Additionally here we're expanding the tuple to include columns
that aren't even likely to be accessed. This is a fairly hot codepath,
and made hugely more expensive by this.
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc return false; /* out of order */ if (att_tup->attisdropped) return false; /* table contains dropped columns */ + if (att_tup->atthasmissing) + return false; /* table contains cols with missing values */
Hm, so incrementally building a table definitions with defaults, even if
there aren't ever any rows, or if there's a subsequent table rewrite,
will cause slowdowns. If so, shouldn't a rewrite remove all
atthasmissing values?
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
@@ -561,10 +568,73 @@ RelationBuildTupleDesc(Relation relation) MemoryContextAllocZero(CacheMemoryContext, relation->rd_rel->relnatts * sizeof(AttrDefault)); - attrdef[ndef].adnum = attp->attnum; + attrdef[ndef].adnum = attnum; attrdef[ndef].adbin = NULL; + ndef++; } + + /* Likewise for a missing value */ + if (attp->atthasmissing) + {
As I've written somewhere above, I don't think it's a good idea to do
this preemptively. Tupledescs are very commonly copied, and the
missing attributes are quite likely never used. IOW, we should just
remember the attmissingattr value and fill in the corresponding
attmissingval on-demand.
+ Datum missingval; + bool missingNull; + + if (attrmiss == NULL) + attrmiss = (AttrMissing *) + MemoryContextAllocZero(CacheMemoryContext, + relation->rd_rel->relnatts * + sizeof(AttrMissing)); + + /* Do we have a missing value? */ + missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple, + Anum_pg_attribute_attmissingval, + &missingNull);
Shouldn't this be a normal heap_getattr rather than a SysCacheGetAttr
access?
* SysCacheGetAttr
*
* Given a tuple previously fetched by SearchSysCache(),
* extract a specific attribute.
@@ -620,7 +691,19 @@ RelationBuildTupleDesc(Relation relation) AttrDefaultFetch(relation); } else + { constr->num_defval = 0; + } + + if (nmissing > 0) + { + if (nmissing < relation->rd_rel->relnatts) + constr->missing = (AttrMissing *) + repalloc(attrmiss, nmissing * sizeof(AttrMissing)); + else + constr->missing = attrmiss; + } + constr->num_missing = nmissing;
Am I understand correctly that you're *downsizing* the allocation with
repalloc here? That seems likely to commonly lead to some wastage...
@@ -4059,10 +4142,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
Hm, it's not obvious why this is a good thing?
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h index 415efbab97..ed30f98e33 100644 --- a/src/include/access/tupdesc.h +++ b/src/include/access/tupdesc.h @@ -14,6 +14,7 @@ #ifndef TUPDESC_H #define TUPDESC_H+#include "postgres.h"
#include "access/attnum.h"
postgres.h should never be included in headers, why was it added here?
This doesn't seem ready yet.
Greetings,
Andres Freund
On Wed, Feb 21, 2018 at 7:48 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
[ Long and useful review]
This doesn't seem ready yet.
Thanks.
I'm working through the issues you raised. Will reply in a few days.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Wed, Feb 21, 2018 at 7:48 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
Comments interspersed.
On 2018-02-20 17:03:07 +1030, Andrew Dunstan wrote:
+ <row> + <entry><structfield>attmissingval</structfield></entry> + <entry><type>bytea</type></entry> + <entry></entry> + <entry> + This column has a binary representation of the value used when the + column is entirely missing from the row, as happens when the column is + added after the row is created. The value is only used when + <structname>atthasmissing</structname> is true. + </entry> + </row>Hm. Why is this a bytea, and why is that a good idea? In pg_statistics,
which has to deal with something similar-ish, we deal with the ambiguity
of the storage by storing anyarrays, which then can display the contents
properly thanks to the embedded oid. Now it'd be a bit sad to store a
one element array in the table. But that seems better than storing a
bytea that's typeless and can't be viewed reasonably. The best solution
would be to create a variadic any type that can actually be stored, but
I see why you'd not want to go there necessarily.
The one-element array idea is a slight hack, but it's not that bad,
and I agree a good deal better than using a bytea. I've made that
change.
But also, a bit higher level, is it a good idea to store this
information directly into pg_attribute? That table is already one of the
hotter system catalog tables, and very frequently the largest. If we
suddenly end up storing a lot of default values in there, possibly ones
that aren't ever actually used because all the default values are long
since actually stored in the table, we'll spend more time doing cache
lookups.I'm somewhat tempted to say that the default data should be in a
separate catalog table. Or possibly use pg_attrdef.
This was debated quite some time ago. See for example
</messages/by-id/22999.1475770835@sss.pgh.pa.us>
The consensus then seemed to be to store them in pg_attribute. If not,
I think we'd need a new catalog - pg_attrdef doesn't seem right. They
would have to go on separate rows, and we'd be storing values rather
than expressions, so it would get somewhat ugly.
+/* + * Return the missing value of an attribute, or NULL if there isn't one. + */ +static Datum +getmissingattr(TupleDesc tupleDesc, + int attnum, bool *isnull) +{ + int missingnum; + Form_pg_attribute att; + AttrMissing *attrmiss; + + Assert(attnum <= tupleDesc->natts); + Assert(attnum > 0); + + att = TupleDescAttr(tupleDesc, attnum - 1); + + if (att->atthasmissing) + { + Assert(tupleDesc->constr); + Assert(tupleDesc->constr->num_missing > 0); + + attrmiss = tupleDesc->constr->missing; + + /* + * Look through the tupledesc's attribute missing values + * for the one that corresponds to this attribute. + */ + for (missingnum = tupleDesc->constr->num_missing - 1; + missingnum >= 0; missingnum--) + { + if (attrmiss[missingnum].amnum == attnum) + { + if (attrmiss[missingnum].ammissingNull) + { + *isnull = true; + return (Datum) 0; + } + else + { + *isnull = false; + return attrmiss[missingnum].ammissing; + } + } + }I think requiring this is a bad idea. If, and only if, there's default
attributes with missing values we should build an array that can be
directly indexed by attribute number, accessing the missing value. What
you're doing here is essentially O(n^2) afaict, with n being the number
of columns missing values.
Done
+/* + * Fill in missing values for a TupleTableSlot + */ +static void +slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum) +{ + AttrMissing *attrmiss; + int missingnum; + int missattnum; + int i; + + /* initialize all the missing attributes to null */ + for (missattnum = lastAttNum - 1; missattnum >= startAttNum; missattnum--) + { + slot->tts_values[missattnum] = PointerGetDatum(NULL); + slot->tts_isnull[missattnum] = true; + }Any reason not to memset this in one (well two) go with memset? Also,
how come you're using PointerGetDatum()? Normally we just use (Datum) 0,
imo it's confusing to use PointerGetDatum(), as it implies things that
are quite possibly not the case.
rewritten.
+/* + * Fill in one attribute, either a data value or a bit in the null bitmask + */Imo this isn't sufficient documentation to explain what this function
does. Even if you just add "per-attribute helper for heap_fill_tuple
and other routines building tuples" or such, I think it'd be clearer.
Changed that way.
+static inline void +fill_val(Form_pg_attribute att, + bits8 **bit, + int *bitmask, + char **dataP, + uint16 *infomask, + Datum datum, + bool isnull) +{ + Size data_length; + char *data = *dataP; + + /* + * If we're building a null bitmap, set the appropriate bit for the + * current column value here. + */ + if (bit != NULL) + { + if (*bitmask != HIGHBIT) + *bitmask <<= 1; + else + { + *bit += 1; + **bit = 0x0; + *bitmask = 1; + } + + if (isnull) + { + *infomask |= HEAP_HASNULL; + return; + } + + **bit |= *bitmask; + } + + Assert(att);Why isn't this asserted at the beginning?
I don't think it's worth very much anyway, so I just removed it.
@@ -293,10 +408,18 @@ heap_fill_tuple(TupleDesc tupleDesc, * ---------------- */ bool -heap_attisnull(HeapTuple tup, int attnum) +heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc) { + /* + * We allow a NULL tupledesc for relations not expected to have + * missing values, such as catalog relations and indexes. + */ + Assert(!tupleDesc || attnum <= tupleDesc->natts);This seems fairly likely to hide bugs.
How? There are plenty of cases where we simply expect quite reasonably
that there won't be any missing attributes, and we don't need a
tupleDesc at all in such cases.
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data)) - return true; + { + return !(tupleDesc && + TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing);Brrr.
rewritten.
+/* + * Expand a tuple with missing values, or NULLs. The source tuple must have + * less attributes than the required number. Only one of targetHeapTuple + * and targetMinimalTuple may be supplied. The other argument must be NULL. + */I can't quite make head or tails of this comment "with missing values, or NULLs"?
Rewritten.
+static void +expand_tuple(HeapTuple *targetHeapTuple, + MinimalTuple *targetMinimalTuple, + HeapTuple sourceTuple, + TupleDesc tupleDesc) +{ + AttrMissing *attrmiss = NULL; + int attnum; + int missingnum; + int firstmissingnum = 0; + int num_missing = 0; + bool hasNulls = HeapTupleHasNulls(sourceTuple); + HeapTupleHeader targetTHeader; + HeapTupleHeader sourceTHeader = sourceTuple->t_data; + int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader); + int natts = tupleDesc->natts; + int sourceIndicatorLen; + int targetIndicatorLen; + Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff; + Size targetDataLen; + Size len; + int hoff; + bits8 *nullBits = NULL; + int bitMask = 0; + char *targetData; + uint16 *infoMask; + + Assert((targetHeapTuple && !targetMinimalTuple) + || (!targetHeapTuple && targetMinimalTuple)); + + Assert(sourceNatts < natts); + + sourceIndicatorLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);Maybe I'm just grumpy right now (did response work on 2016 taxes just
now, brrrrr), but I don't find "indicatorLen" a descriptive term.
changed
+ targetDataLen = sourceDataLen; + + if (tupleDesc->constr && + tupleDesc->constr->num_missing) + { + /* + * If there are missing values we want to put them into the tuple. + * Before that we have to compute the extra length for the values + * array and the variable length data. + */ + num_missing = tupleDesc->constr->num_missing; + attrmiss = tupleDesc->constr->missing;You're accessing a hell of a lot of expensive stuff outside outside of
this if block. E.g. HeapTupleHeaderGetNatts specifically is the prime
source of cache misses in a lot of workloads. Now a smart compiler might
be able to optimize this away, but...+ /* + * Find the first item in attrmiss for which we don't have a value in + * the source (i.e. where its amnum is > sourceNatts). We can ignore + * all the missing entries before that. + */ + for (firstmissingnum = 0; + firstmissingnum < num_missing + && attrmiss[firstmissingnum].amnum <= sourceNatts; + firstmissingnum++) + { /* empty */ + } + + /* + * If there are no more missing values everything else must be + * NULL + */ + if (firstmissingnum >= num_missing) + { + hasNulls = true; + } + + /* + * Now walk the missing attributes one by one. If there is a + * missing value make space for it. Otherwise, it's going to be NULL. + */ + for (attnum = sourceNatts + 1; + attnum <= natts; + attnum++) + { + Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum - 1); + + for (missingnum = firstmissingnum; missingnum < num_missing; missingnum++) + { + + /* + * If there is a missing value for this attribute calculate + * the required space if it's not NULL. + */ + if (attrmiss[missingnum].amnum == attnum) + { + if (attrmiss[missingnum].ammissingNull) + hasNulls = true; + else + { + targetDataLen = att_align_datum(targetDataLen, + att->attalign, + att->attlen, + attrmiss[missingnum].ammissing); + + targetDataLen = att_addlength_pointer(targetDataLen, + att->attlen, + attrmiss[missingnum].ammissing); + } + break; + } + } + if (missingnum > num_missing) + { + /* there is no missing value for this attribute */ + hasNulls = true; + } + } + } /* end if have missing values */ + else + { + /* + * If there are no missing values at all then NULLS must be allowed, + * since some of the attributes are known to be absent. + */ + hasNulls = true;Why are we doing anything at all in this branch? ISTM we're reforming
the tuple for absolutely no gain here, just making it larger by
including additionall null bits?+ /* + * Allocate and zero the space needed. Note that the tuple body and + * HeapTupleData management structure are allocated in one chunk. + */ + if (targetHeapTuple) + { + len += offsetof(HeapTupleHeaderData, t_bits); + hoff = len = MAXALIGN(len); /* align user data safely */ + len += targetDataLen;So len isn't the length of the tuple, but the null bitmap length? That
seems, uh, non-obvious.+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len); + (*targetHeapTuple)->t_data + = targetTHeader + = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);Yuck.
+ (*targetHeapTuple)->t_len = len; + (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid; + ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));Seems like a lot of this code would be more readable by storing
*targetHeapTuple in a local var.+ targetTHeader->t_infomask = sourceTHeader->t_infomask; + targetTHeader->t_hoff = hoff; + HeapTupleHeaderSetNatts(targetTHeader, natts); + HeapTupleHeaderSetDatumLength(targetTHeader, len); + HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid); + HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod); + /* We also make sure that t_ctid is invalid unless explicitly set */ + ItemPointerSetInvalid(&(targetTHeader->t_ctid));This seems fairly expensive, why aren't we just memcpy'ing the old
header into the new one?+ if (targetIndicatorLen > 0) + { + if (sourceIndicatorLen > 0) + { + /* if bitmap pre-existed copy in - all is set */can't parse.
+ memcpy(nullBits, + ((char *) sourceTHeader) + + offsetof(HeapTupleHeaderData, t_bits), + sourceIndicatorLen); + nullBits += sourceIndicatorLen - 1; + } + else + { + sourceIndicatorLen = BITMAPLEN(sourceNatts); + /* Set NOT NULL for all existing attributes */ + memset(nullBits, 0xff, sourceIndicatorLen); + + nullBits += sourceIndicatorLen - 1; + + if (sourceNatts & 0x07) + { + /* build the mask (inverted!) */ + bitMask = 0xff << (sourceNatts & 0x07); + /* Voila */ + *nullBits = ~bitMask; + }It's late, I'm tired. But how can this be correct? Also, this code is
massively under-commented.
I'm going to revisit expand_tuple from scratch.
+ +/* + * Fill in the missing values for an ordinary HeapTuple + */ +MinimalTuple +minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc) +{ + MinimalTuple minimalTuple; + + expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc); + return minimalTuple; +} + +/* + * Fill in the missing values for a minimal HeapTuple + */ +HeapTuple +heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc) +{ + HeapTuple heapTuple; + + expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc); + return heapTuple; +}Comments appear to be swapped.
Fixed.
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1422,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,/* * If tuple doesn't have all the atts indicated by tupleDesc, read the - * rest as null + * rest as null, or a missing value if there is one. */"a missing value" isn't there quite possibly multiple ones?
Rewritten
/* @@ -1192,8 +1599,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull) tup = tuple->t_data; if (attnum > HeapTupleHeaderGetNatts(tup)) { - *isnull = true; - return (Datum) 0; + return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull); }This turned a ~4 instruction case into considerably more. I've recently
removed one of the major slot_getattr sources (hashagg comparisons), but
we still call this super frequently for hashagg's hash computations. I
suggest micro-benchmarking that case.
I'll look at it. I have rewritten getmissingattr along the line you've
suggested, so it should be more efficient, and most compilers will
inline it.
Oid StoreAttrDefault(Relation rel, AttrNumber attnum, - Node *expr, bool is_internal) + Node *expr, bool is_internal, bool add_column_mode) { char *adbin; char *adsrc; @@ -1939,9 +1945,20 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, Relation attrrel; HeapTuple atttup; Form_pg_attribute attStruct; + Form_pg_attribute defAttStruct; Oid attrdefOid; ObjectAddress colobject, defobject; + ExprState *exprState; + Expr *expr2 = (Expr *) expr; + EState *estate = NULL; + ExprContext *econtext; + char *missingBuf = NULL; + Datum valuesAtt[Natts_pg_attribute]; + bool nullsAtt[Natts_pg_attribute]; + bool replacesAtt[Natts_pg_attribute]; + Datum missingval = (Datum) 0; + bool missingIsNull = true;/*
* Flatten expression to string form for storage.
@@ -1956,6 +1973,44 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);+ /* + * Compute the missing value + */ + expr2 = expression_planner(expr2); + + exprState = ExecInitExpr(expr2, NULL);This should instead use ExecPrepareExpr()
Fixed
+ if (add_column_mode) + { + missingval = ExecEvalExpr(exprState, econtext, + &missingIsNull); + + defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1); + + if (missingIsNull) + { + missingval = PointerGetDatum(NULL);(Datum) 0;
+ } + else if (defAttStruct->attbyval) + { + missingBuf = palloc(VARHDRSZ + sizeof(Datum)); + memcpy(VARDATA(missingBuf), &missingval, sizeof(Datum)); + SET_VARSIZE(missingBuf, VARHDRSZ + sizeof(Datum)); + missingval = PointerGetDatum(missingBuf); + } + else if (defAttStruct->attlen >= 0) + { + missingBuf = palloc(VARHDRSZ + defAttStruct->attlen); + memcpy(VARDATA(missingBuf), DatumGetPointer(missingval), + defAttStruct->attlen); + SET_VARSIZE(missingBuf, + VARHDRSZ + defAttStruct->attlen); + missingval = PointerGetDatum(missingBuf); + }What if we get an extended datum back in the varlena case? Even if we
wouldn't change the storage from bytea to something else, you really
would need to force detoasting here.
This is now rewritten.
+ } + /* * Make the pg_attrdef entry. */ @@ -1996,7 +2051,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum, attStruct = (Form_pg_attribute) GETSTRUCT(atttup); if (!attStruct->atthasdef) { - attStruct->atthasdef = true; + MemSet(valuesAtt, 0, sizeof(valuesAtt)); + MemSet(nullsAtt, false, sizeof(nullsAtt)); + MemSet(replacesAtt, false, sizeof(replacesAtt)); + valuesAtt[Anum_pg_attribute_atthasdef - 1] = true; + replacesAtt[Anum_pg_attribute_atthasdef - 1] = true; + if (add_column_mode) + { + valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true; + replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true; + valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval; + replacesAtt[Anum_pg_attribute_attmissingval - 1] = true; + nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull; + } + atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel), + valuesAtt, nullsAtt, replacesAtt); +This seems weird. We've above formed a tuple, now we just modify it
again?
I'll take a look at making it more straightforward. But It's not as
simple as the atthasdef case, since attmissing is part of the variable
portion of pg_attribute.
@@ -2296,7 +2377,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal); + /* If the default is volatile we cannot use a missing value */ + if (contain_volatile_functions((Node *) expr)) + { + colDef->missingMode = false; + } + defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal, + colDef->missingMode);Doesn't this have to be immutable rather than just stable? I mean just
storing the value of now() doesn't seem like it'd do the trick?
I don't see why. We've stated explicitly in the docs that the existing
rows get the value of the default expression as evaluated at the time
the column is created. I don't think there is any need to restrict
that to an immutable expression.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot) * If we have a regular physical tuple then just return it. */ if (TTS_HAS_PHYSICAL_TUPLE(slot)) - return slot->tts_tuple; + { + if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) < + slot->tts_tupleDescriptor->natts) + { + MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt); + + slot->tts_tuple = heap_expand_tuple(slot->tts_tuple, + slot->tts_tupleDescriptor); + slot->tts_shouldFree = true; + MemoryContextSwitchTo(oldContext); + return slot->tts_tuple; + } + else + { + return slot->tts_tuple; + } + }I'm extremely dubious about this. HeapTupleHeaderGetNatts() commonly
causes cachemisses and therefore causes slowdowns when unnecessarily
done. Additionally here we're expanding the tuple to include columns
that aren't even likely to be accessed. This is a fairly hot codepath,
and made hugely more expensive by this.
Will look further.
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc return false; /* out of order */ if (att_tup->attisdropped) return false; /* table contains dropped columns */ + if (att_tup->atthasmissing) + return false; /* table contains cols with missing values */Hm, so incrementally building a table definitions with defaults, even if
there aren't ever any rows, or if there's a subsequent table rewrite,
will cause slowdowns. If so, shouldn't a rewrite remove all
atthasmissing values?
Yes, I think so. Working on it. OTOH, I'm somewhat mystified by having
had to make this change. Still, it looks like dropping a column has
the same effect.
@@ -493,7 +494,10 @@ RelationBuildTupleDesc(Relation relation)
@@ -561,10 +568,73 @@ RelationBuildTupleDesc(Relation relation) MemoryContextAllocZero(CacheMemoryContext, relation->rd_rel->relnatts * sizeof(AttrDefault)); - attrdef[ndef].adnum = attp->attnum; + attrdef[ndef].adnum = attnum; attrdef[ndef].adbin = NULL; + ndef++; } + + /* Likewise for a missing value */ + if (attp->atthasmissing) + {As I've written somewhere above, I don't think it's a good idea to do
this preemptively. Tupledescs are very commonly copied, and the
missing attributes are quite likely never used. IOW, we should just
remember the attmissingattr value and fill in the corresponding
attmissingval on-demand.
Defaults are frequently not used either, yet this function fills those
in regardless. I don't see why we need to treat missing values
differently.
+ Datum missingval; + bool missingNull; + + if (attrmiss == NULL) + attrmiss = (AttrMissing *) + MemoryContextAllocZero(CacheMemoryContext, + relation->rd_rel->relnatts * + sizeof(AttrMissing)); + + /* Do we have a missing value? */ + missingval = SysCacheGetAttr(ATTNUM, pg_attribute_tuple, + Anum_pg_attribute_attmissingval, + &missingNull);Shouldn't this be a normal heap_getattr rather than a SysCacheGetAttr
access?* SysCacheGetAttr
*
* Given a tuple previously fetched by SearchSysCache(),
* extract a specific attribute.
Fixed.
@@ -620,7 +691,19 @@ RelationBuildTupleDesc(Relation relation) AttrDefaultFetch(relation); } else + { constr->num_defval = 0; + } + + if (nmissing > 0) + { + if (nmissing < relation->rd_rel->relnatts) + constr->missing = (AttrMissing *) + repalloc(attrmiss, nmissing * sizeof(AttrMissing)); + else + constr->missing = attrmiss; + } + constr->num_missing = nmissing;Am I understand correctly that you're *downsizing* the allocation with
repalloc here? That seems likely to commonly lead to some wastage...
This code is gone. But note the similar code nearby that is already there.
@@ -4059,10 +4142,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}Hm, it's not obvious why this is a good thing?
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h index 415efbab97..ed30f98e33 100644 --- a/src/include/access/tupdesc.h +++ b/src/include/access/tupdesc.h @@ -14,6 +14,7 @@ #ifndef TUPDESC_H #define TUPDESC_H+#include "postgres.h"
#include "access/attnum.h"postgres.h should never be included in headers, why was it added here?
Fixed.
This doesn't seem ready yet.
I hope we're a lot closer now.
I'm working on rejigging expand_tuple and on clearing the missing
attributes on a table rewrite.
Attached is the current state of things, with the fixes indicated
above, as well as some things pointed out by David Rowley.
None of this seems to have had much effect on performance.
Here's what oprofile reports from a run of pgbench with
create-alter.sql and the query "select sum(c1000) from t":
Profiling through timer interrupt
samples % image name symbol name
22584 28.5982 postgres ExecInterpExpr
11950 15.1323 postgres slot_getmissingattrs
5471 6.9279 postgres AllocSetAlloc
3018 3.8217 postgres TupleDescInitEntry
2042 2.5858 postgres MemoryContextAllocZeroAligned
1807 2.2882 postgres SearchCatCache1
1638 2.0742 postgres expression_tree_walker
That's very different from what we see on the master branch. The time
spent in slot_getmissingattrs is perhaps not unexpected, but I don't
(at least yet) understand why we're getting so much time spent in
ExecInterpExpr, which doesn't show up at all when the same benchmark
is run on master.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v12.patchapplication/octet-stream; name=fast_default-v12.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..8bcd5b8b19 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,19 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>anyarray</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a one element array containing the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true. If there is no value
+ the column is null.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..edf459ae6b 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1184,24 +1184,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..9fc178c123 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -58,6 +58,7 @@
#include "postgres.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -76,6 +77,68 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->missing);
+
+ attrmiss = tupleDesc->constr->missing + (attnum - 1);
+
+ if (attrmiss->ammissingPresent)
+ {
+ *isnull = false;
+ return attrmiss->ammissing;
+ }
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss = NULL;
+ int missattnum;
+
+ memset(slot->tts_values + startAttNum, 0,
+ (lastAttNum - startAttNum) * sizeof(Datum));
+ memset(slot->tts_isnull + startAttNum, 1,
+ (lastAttNum - startAttNum) * sizeof(bool));
+
+ if (slot->tts_tupleDescriptor->constr)
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+
+ if(!attrmiss)
+ return;
+
+ for (missattnum = lastAttNum - 1; missattnum >= startAttNum; missattnum--)
+ {
+ if (attrmiss[missattnum].ammissingPresent)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +195,134 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Per-attribute helper for heap_fill_tuple and other routines building tuples.
+ *
+ * Fill in either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+
+
+
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +363,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +388,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if(tupleDesc && TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ return false;
+ else
+ return true;
+ }
if (attnum > 0)
{
@@ -649,6 +754,276 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple which has less attributes than required. For each attribute
+ * not present in the sourceTuple, if there is a missing value that will be
+ * used. Otherwise the attribute will be set to NULL.
+ *
+ * The source tuple must have less attributes than the required number.
+ *
+ * Only one of targetHeapTuple and targetMinimalTuple may be supplied. The
+ * other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int firstmissingnum = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceNullLen;
+ int targetNullLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceNullLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = sourceNatts;
+ firstmissingnum < natts;
+ firstmissingnum++)
+ {
+ if (attrmiss[firstmissingnum].ammissingPresent)
+ break;
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= natts)
+ {
+ hasNulls = true;
+ }
+ else
+ {
+
+ /*
+ * Now walk the missing attributes. If there is a missing value
+ * make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = firstmissingnum;
+ attnum < natts;
+ attnum++)
+ {
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum);
+
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+ }
+ else
+ {
+ /* no missing value, so it must be null */
+ hasNulls = true;
+ }
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetNullLen = BITMAPLEN(natts);
+ len += targetNullLen;
+ }
+ else
+ targetNullLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetNullLen > 0)
+ {
+ if (sourceNullLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceNullLen);
+ nullBits += sourceNullLen - 1;
+ }
+ else
+ {
+ sourceNullLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceNullLen);
+
+ nullBits += sourceNullLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ targetData += sourceDataLen;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts ; attnum < natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[attnum].ammissing,
+ false);
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1387,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as nulls or missing values as appropriate.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1564,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1634,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1707,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1730,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..2b7d53ae2f 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +178,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if(constr->missing)
+ {
+ cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissingPresent)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->missing)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissingPresent
+ && !TupleDescAttr(tupdesc, i)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +500,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
+
+ if (missval1->ammissingPresent != missval2->ammissingPresent)
+ return false;
+ if (missval1->ammissingPresent)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ }
+ }
+ else if (constr2->missing)
+ return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +638,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,6 +852,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..0aaa3e3ba9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1928,7 +1934,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +1945,19 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +1972,40 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ estate = CreateExecutorState();
+ exprState = ExecPrepareExpr(expr2, estate);
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ /* if the default evaluates to NULL, just store a NULL array */
+ missingval = (Datum) 0;
+ }
+ else
+ {
+ /* otherwise make a one-element array of the value */
+ missingval = PointerGetDatum(
+ construct_array(&missingval,
+ 1,
+ defAttStruct->atttypid,
+ defAttStruct->attlen,
+ defAttStruct->attbyval,
+ defAttStruct->attalign));
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2046,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2093,14 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
return attrdefOid;
}
@@ -2180,7 +2253,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2369,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 564f2069cf..48bac61991 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1668,7 +1669,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..aa654067ba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index abdfa249c0..361033e185 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2257,7 +2257,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 3e48a58dcb..4f2bbe12b4 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 74e020bffc..0992f568d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4681,7 +4682,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4784,7 +4785,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5403,6 +5404,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5446,6 +5448,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5456,6 +5459,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5501,6 +5513,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5536,16 +5551,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6021,6 +6041,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8107,8 +8128,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9514,7 +9535,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 771b7e3945..e1c1fb741a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2483,7 +2483,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 91ba939bdc..aa5c7cbe0e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf93..025d3e379c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a8ae37ebc8..7012a2e8b3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..361bde4261 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3697466789..38c3b3f06b 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1240,7 +1240,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1408,7 +1408,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1642,7 +1642,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..bf7e001699 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -36,6 +36,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +495,9 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,8 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +560,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +568,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
@@ -605,7 +665,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ attrmiss || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +681,10 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+ constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4123,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4461,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4756,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4819,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..6d8051b850 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -25,6 +25,8 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing *MissingPtr;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,6 +40,7 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ MissingPtr missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
bool has_not_null;
diff --git a/src/include/access/tupdesc_details.h b/src/include/access/tupdesc_details.h
new file mode 100644
index 0000000000..c5d9605974
--- /dev/null
+++ b/src/include/access/tupdesc_details.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * tupdesc_details.h
+ * POSTGRES tuple descriptor definitions we can't include everywhere
+ *
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/tupdesc_details.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TUPDESC_DETAILS_H
+#define TUPDESC_DETAILS_H
+
+/*
+ * Structure used to represent value to be used when the attribute is not
+ * present at all in a tuple, i.e. when the column was created after the tuple
+ */
+
+typedef struct attrMissing
+{
+ bool ammissingPresent; /* true if non-NULL missing value exists */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
+#endif /* TUPDESC_DETAILS_H */
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..5869a487da 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,7 +104,8 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_internal);
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..280e36c5d2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +200,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +215,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..9e74068a0a
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..c4bf9d61e4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing
+test: identity partition_join partition_prune reloptions hash_part indexing fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..6a3f9b367e
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
Hi,
On 2018-02-27 14:29:44 +1030, Andrew Dunstan wrote:
This was debated quite some time ago. See for example
</messages/by-id/22999.1475770835@sss.pgh.pa.us>
The consensus then seemed to be to store them in pg_attribute. If not,
I think we'd need a new catalog - pg_attrdef doesn't seem right. They
would have to go on separate rows, and we'd be storing values rather
than expressions, so it would get somewhat ugly.
I know that I'm concerned with storing a number of large values in
pg_attributes, and I haven't seen that argument addressed much in a
quick scan of the discussion.
@@ -293,10 +408,18 @@ heap_fill_tuple(TupleDesc tupleDesc, * ---------------- */ bool -heap_attisnull(HeapTuple tup, int attnum) +heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc) { + /* + * We allow a NULL tupledesc for relations not expected to have + * missing values, such as catalog relations and indexes. + */ + Assert(!tupleDesc || attnum <= tupleDesc->natts);This seems fairly likely to hide bugs.
How? There are plenty of cases where we simply expect quite reasonably
that there won't be any missing attributes, and we don't need a
tupleDesc at all in such cases.
Missing to pass a tupledesc for tables that can have defaults with not
have directly visible issues, you'll just erroneously get back a null.
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc return false; /* out of order */ if (att_tup->attisdropped) return false; /* table contains dropped columns */ + if (att_tup->atthasmissing) + return false; /* table contains cols with missing values */Hm, so incrementally building a table definitions with defaults, even if
there aren't ever any rows, or if there's a subsequent table rewrite,
will cause slowdowns. If so, shouldn't a rewrite remove all
atthasmissing values?
Yes, I think so. Working on it. OTOH, I'm somewhat mystified by having
had to make this change. Still, it looks like dropping a column has
the same effect.
What exactly is mystifying you? That the new default stuff doesn't work
when the physical tlist optimization is in use?
@@ -561,10 +568,73 @@ RelationBuildTupleDesc(Relation relation) MemoryContextAllocZero(CacheMemoryContext, relation->rd_rel->relnatts * sizeof(AttrDefault)); - attrdef[ndef].adnum = attp->attnum; + attrdef[ndef].adnum = attnum; attrdef[ndef].adbin = NULL; + ndef++; } + + /* Likewise for a missing value */ + if (attp->atthasmissing) + {As I've written somewhere above, I don't think it's a good idea to do
this preemptively. Tupledescs are very commonly copied, and the
missing attributes are quite likely never used. IOW, we should just
remember the attmissingattr value and fill in the corresponding
attmissingval on-demand.
Defaults are frequently not used either, yet this function fills those
in regardless. I don't see why we need to treat missing values
differently.
It's already hurting us quite a bit in workloads with a lot of trivial
queries. Adding more makes it worse.
Attached is the current state of things, with the fixes indicated
above, as well as some things pointed out by David Rowley.None of this seems to have had much effect on performance.
Here's what oprofile reports from a run of pgbench with
create-alter.sql and the query "select sum(c1000) from t":Profiling through timer interrupt
samples % image name symbol name
22584 28.5982 postgres ExecInterpExpr
11950 15.1323 postgres slot_getmissingattrs
5471 6.9279 postgres AllocSetAlloc
3018 3.8217 postgres TupleDescInitEntry
2042 2.5858 postgres MemoryContextAllocZeroAligned
1807 2.2882 postgres SearchCatCache1
1638 2.0742 postgres expression_tree_walkerThat's very different from what we see on the master branch. The time
spent in slot_getmissingattrs is perhaps not unexpected, but I don't
(at least yet) understand why we're getting so much time spent in
ExecInterpExpr, which doesn't show up at all when the same benchmark
is run on master.
I'd guess that's because the physical tlist optimization is disabled. I
assume you'd see something similar on master if you dropped a column.
- Andres
On Wed, Feb 28, 2018 at 5:39 AM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-02-27 14:29:44 +1030, Andrew Dunstan wrote:
This was debated quite some time ago. See for example
</messages/by-id/22999.1475770835@sss.pgh.pa.us>
The consensus then seemed to be to store them in pg_attribute. If not,
I think we'd need a new catalog - pg_attrdef doesn't seem right. They
would have to go on separate rows, and we'd be storing values rather
than expressions, so it would get somewhat ugly.I know that I'm concerned with storing a number of large values in
pg_attributes, and I haven't seen that argument addressed much in a
quick scan of the discussion.
I'd like to have a consensus on this before I start making such a
change as adding a new catalog table.
@@ -293,10 +408,18 @@ heap_fill_tuple(TupleDesc tupleDesc, * ---------------- */ bool -heap_attisnull(HeapTuple tup, int attnum) +heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc) { + /* + * We allow a NULL tupledesc for relations not expected to have + * missing values, such as catalog relations and indexes. + */ + Assert(!tupleDesc || attnum <= tupleDesc->natts);This seems fairly likely to hide bugs.
How? There are plenty of cases where we simply expect quite reasonably
that there won't be any missing attributes, and we don't need a
tupleDesc at all in such cases.Missing to pass a tupledesc for tables that can have defaults with not
have directly visible issues, you'll just erroneously get back a null.
What would you prefer? Two versions of the function?
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc return false; /* out of order */ if (att_tup->attisdropped) return false; /* table contains dropped columns */ + if (att_tup->atthasmissing) + return false; /* table contains cols with missing values */Hm, so incrementally building a table definitions with defaults, even if
there aren't ever any rows, or if there's a subsequent table rewrite,
will cause slowdowns. If so, shouldn't a rewrite remove all
atthasmissing values?Yes, I think so. Working on it. OTOH, I'm somewhat mystified by having
had to make this change. Still, it looks like dropping a column has
the same effect.What exactly is mystifying you? That the new default stuff doesn't work
when the physical tlist optimization is in use?
I think I have a better handle on that now. I'm trying to think of a
way around it, no massive inspiration yet.
@@ -561,10 +568,73 @@ RelationBuildTupleDesc(Relation relation) MemoryContextAllocZero(CacheMemoryContext, relation->rd_rel->relnatts * sizeof(AttrDefault)); - attrdef[ndef].adnum = attp->attnum; + attrdef[ndef].adnum = attnum; attrdef[ndef].adbin = NULL; + ndef++; } + + /* Likewise for a missing value */ + if (attp->atthasmissing) + {As I've written somewhere above, I don't think it's a good idea to do
this preemptively. Tupledescs are very commonly copied, and the
missing attributes are quite likely never used. IOW, we should just
remember the attmissingattr value and fill in the corresponding
attmissingval on-demand.Defaults are frequently not used either, yet this function fills those
in regardless. I don't see why we need to treat missing values
differently.It's already hurting us quite a bit in workloads with a lot of trivial
queries. Adding more makes it worse.
Please expand on your view of how we should "just remember the
attmissingattr value and fill in the corresponding attmissingval
on-demand."
I don't mind changing how we do things, but I need a bit more detail than this.
If copying defaults around has been hurting us so much it seems
surprising nobody has put forward an improvement.
Attached is the current state of things, with the fixes indicated
above, as well as some things pointed out by David Rowley.None of this seems to have had much effect on performance.
Here's what oprofile reports from a run of pgbench with
create-alter.sql and the query "select sum(c1000) from t":Profiling through timer interrupt
samples % image name symbol name
22584 28.5982 postgres ExecInterpExpr
11950 15.1323 postgres slot_getmissingattrs
5471 6.9279 postgres AllocSetAlloc
3018 3.8217 postgres TupleDescInitEntry
2042 2.5858 postgres MemoryContextAllocZeroAligned
1807 2.2882 postgres SearchCatCache1
1638 2.0742 postgres expression_tree_walkerThat's very different from what we see on the master branch. The time
spent in slot_getmissingattrs is perhaps not unexpected, but I don't
(at least yet) understand why we're getting so much time spent in
ExecInterpExpr, which doesn't show up at all when the same benchmark
is run on master.I'd guess that's because the physical tlist optimization is disabled. I
assume you'd see something similar on master if you dropped a column.
Yeah. That's kinda ugly in fact.
Here's a version of the patch which cleans up the missing attribute
settings on a table rewrite.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v13.patchapplication/octet-stream; name=fast_default-v13.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..8bcd5b8b19 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structname>attmissingval</structname> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,19 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>anyarray</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a one element array containing the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structname>atthasmissing</structname> is true. If there is no value
+ the column is null.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..edf459ae6b 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1184,24 +1184,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..9fc178c123 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -58,6 +58,7 @@
#include "postgres.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -76,6 +77,68 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->missing);
+
+ attrmiss = tupleDesc->constr->missing + (attnum - 1);
+
+ if (attrmiss->ammissingPresent)
+ {
+ *isnull = false;
+ return attrmiss->ammissing;
+ }
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss = NULL;
+ int missattnum;
+
+ memset(slot->tts_values + startAttNum, 0,
+ (lastAttNum - startAttNum) * sizeof(Datum));
+ memset(slot->tts_isnull + startAttNum, 1,
+ (lastAttNum - startAttNum) * sizeof(bool));
+
+ if (slot->tts_tupleDescriptor->constr)
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+
+ if(!attrmiss)
+ return;
+
+ for (missattnum = lastAttNum - 1; missattnum >= startAttNum; missattnum--)
+ {
+ if (attrmiss[missattnum].ammissingPresent)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] = false;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +195,134 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Per-attribute helper for heap_fill_tuple and other routines building tuples.
+ *
+ * Fill in either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+
+
+
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +363,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +388,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if(tupleDesc && TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ return false;
+ else
+ return true;
+ }
if (attnum > 0)
{
@@ -649,6 +754,276 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple which has less attributes than required. For each attribute
+ * not present in the sourceTuple, if there is a missing value that will be
+ * used. Otherwise the attribute will be set to NULL.
+ *
+ * The source tuple must have less attributes than the required number.
+ *
+ * Only one of targetHeapTuple and targetMinimalTuple may be supplied. The
+ * other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int firstmissingnum = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceNullLen;
+ int targetNullLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceNullLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source (i.e. where its amnum is > sourceNatts). We can ignore
+ * all the missing entries before that.
+ */
+ for (firstmissingnum = sourceNatts;
+ firstmissingnum < natts;
+ firstmissingnum++)
+ {
+ if (attrmiss[firstmissingnum].ammissingPresent)
+ break;
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= natts)
+ {
+ hasNulls = true;
+ }
+ else
+ {
+
+ /*
+ * Now walk the missing attributes. If there is a missing value
+ * make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = firstmissingnum;
+ attnum < natts;
+ attnum++)
+ {
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum);
+
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+ }
+ else
+ {
+ /* no missing value, so it must be null */
+ hasNulls = true;
+ }
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetNullLen = BITMAPLEN(natts);
+ len += targetNullLen;
+ }
+ else
+ targetNullLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetNullLen > 0)
+ {
+ if (sourceNullLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceNullLen);
+ nullBits += sourceNullLen - 1;
+ }
+ else
+ {
+ sourceNullLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceNullLen);
+
+ nullBits += sourceNullLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ targetData += sourceDataLen;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts ; attnum < natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[attnum].ammissing,
+ false);
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1387,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as nulls or missing values as appropriate.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1564,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1634,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1707,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1730,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..2b7d53ae2f 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +178,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if(constr->missing)
+ {
+ cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissingPresent)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->missing)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissingPresent
+ && !TupleDescAttr(tupdesc, i)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +500,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
+
+ if (missval1->ammissingPresent != missval2->ammissingPresent)
+ return false;
+ if (missval1->ammissingPresent)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ }
+ }
+ else if (constr2->missing)
+ return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +638,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,6 +852,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 1156627b9e..e88a6d5f89 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4592,7 +4592,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..73fe9bed57 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1921,6 +1927,75 @@ heap_drop_with_catalog(Oid relid)
}
+/*
+ * RelationClearMissing
+ *
+ * set atthasmissing and attmissingval to false/null for all attributes
+ * where they are currently set. This can be safely and usefully done if
+ * the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
+ * are no rows left with less than a full complement of attributes.
+ */
+
+void
+RelationClearMissing(Relation rel)
+{
+ Relation attr_rel;
+ Oid relid = RelationGetRelid(rel);
+ int natts = RelationGetNumberOfAttributes(rel);
+ int attnum;
+ Datum repl_val[Natts_pg_attribute];
+ bool repl_null[Natts_pg_attribute];
+ bool repl_repl[Natts_pg_attribute];
+ Form_pg_attribute attrtuple;
+ HeapTuple tuple,
+ newtuple;
+
+ memset(repl_val, 0, sizeof(repl_val));
+ memset(repl_null, false, sizeof(repl_null));
+ memset(repl_repl, false, sizeof(repl_repl));
+
+ repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false);
+ repl_null[Anum_pg_attribute_attmissingval - 1] = true;
+
+ repl_repl[Anum_pg_attribute_atthasmissing - 1] = true;
+ repl_repl[Anum_pg_attribute_attmissingval - 1] = true;
+
+
+ /* Get a lock on pg_attribute */
+ attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ /* process each non-system attribute */
+ for (attnum = 1; attnum <= natts; attnum++)
+ {
+ tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+ if (!HeapTupleIsValid(tuple)) /* shouldn't happen */
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /* ignore any where atthasmissing is not true */
+ if (attrtuple->atthasmissing)
+ {
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attr_rel),
+ repl_val, repl_null, repl_repl);
+
+ CatalogTupleUpdate(attr_rel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+
+ ReleaseSysCache(tuple);
+ }
+ /*
+ * Our update of the pg_attribute rows will force a relcache rebuild, so
+ * there's nothing else to do here.
+ */
+ heap_close(attr_rel, RowExclusiveLock);
+}
+
/*
* Store a default expression for column attnum of relation rel.
*
@@ -1928,7 +2003,7 @@ heap_drop_with_catalog(Oid relid)
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +2014,19 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1956,6 +2041,40 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
RelationGetRelid(rel)),
false, false);
+ /*
+ * Compute the missing value
+ */
+ expr2 = expression_planner(expr2);
+
+ estate = CreateExecutorState();
+ exprState = ExecPrepareExpr(expr2, estate);
+ econtext = GetPerTupleExprContext(estate);
+
+ if (add_column_mode)
+ {
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ /* if the default evaluates to NULL, just store a NULL array */
+ missingval = (Datum) 0;
+ }
+ else
+ {
+ /* otherwise make a one-element array of the value */
+ missingval = PointerGetDatum(
+ construct_array(&missingval,
+ 1,
+ defAttStruct->atttypid,
+ defAttStruct->attlen,
+ defAttStruct->attbyval,
+ defAttStruct->attalign));
+ }
+ }
+
/*
* Make the pg_attrdef entry.
*/
@@ -1996,7 +2115,22 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ if (add_column_mode)
+ {
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2162,14 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
return attrdefOid;
}
@@ -2180,7 +2322,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2438,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 564f2069cf..48bac61991 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1668,7 +1669,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..2428b50dba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
@@ -1668,6 +1668,16 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
relation_close(newrel, NoLock);
}
+
+ /* if it's not a catalog table, clear any missing attribute settings */
+ if (!is_system_catalog)
+ {
+ Relation newrel;
+
+ newrel = heap_open(OIDOldHeap, NoLock);
+ RelationClearMissing(newrel);
+ relation_close(newrel, NoLock);
+ }
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index abdfa249c0..361033e185 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2257,7 +2257,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 3e48a58dcb..4f2bbe12b4 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 74e020bffc..0992f568d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4681,7 +4682,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4784,7 +4785,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5403,6 +5404,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5446,6 +5448,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5456,6 +5459,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5501,6 +5513,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5536,16 +5551,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6021,6 +6041,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8107,8 +8128,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9514,7 +9535,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 899a5c4cd4..982b8856f4 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2398,7 +2398,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 771b7e3945..e1c1fb741a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2483,7 +2483,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 91ba939bdc..aa5c7cbe0e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf93..025d3e379c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a8ae37ebc8..7012a2e8b3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 89f27ce0eb..2fefa794e0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->proretset ||
funcform->prorettype == InvalidOid ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..361bde4261 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3697466789..38c3b3f06b 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1240,7 +1240,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1408,7 +1408,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1642,7 +1642,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..bf7e001699 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -36,6 +36,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,7 +495,9 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
+ int attnum = 0;
/* copy some fields from pg_class row to rd_att */
relation->rd_att->tdtypeid = relation->rd_rel->reltype;
@@ -546,6 +550,8 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
+ attnum = attp->attnum;
+
memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +560,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +568,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
@@ -605,7 +665,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ attrmiss || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +681,10 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+ constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4059,10 +4123,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4401,7 +4461,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4696,7 +4756,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4759,7 +4819,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..6d8051b850 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -25,6 +25,8 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing *MissingPtr;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,6 +40,7 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ MissingPtr missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
bool has_not_null;
diff --git a/src/include/access/tupdesc_details.h b/src/include/access/tupdesc_details.h
new file mode 100644
index 0000000000..c5d9605974
--- /dev/null
+++ b/src/include/access/tupdesc_details.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * tupdesc_details.h
+ * POSTGRES tuple descriptor definitions we can't include everywhere
+ *
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/tupdesc_details.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TUPDESC_DETAILS_H
+#define TUPDESC_DETAILS_H
+
+/*
+ * Structure used to represent value to be used when the attribute is not
+ * present at all in a tuple, i.e. when the column was created after the tuple
+ */
+
+typedef struct attrMissing
+{
+ bool ammissingPresent; /* true if non-NULL missing value exists */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
+#endif /* TUPDESC_DETAILS_H */
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..69adbc5d1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -102,8 +103,11 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_local,
bool is_internal);
+extern void RelationClearMissing(Relation rel);
+
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..280e36c5d2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +200,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +215,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..6155caf58b 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..9e74068a0a
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..c4bf9d61e4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing
+test: identity partition_join partition_prune reloptions hash_part indexing fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..6a3f9b367e
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 1 March 2018 at 11:49, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
On Wed, Feb 28, 2018 at 5:39 AM, Andres Freund <andres@anarazel.de> wrote:
On 2018-02-27 14:29:44 +1030, Andrew Dunstan wrote:
Profiling through timer interrupt
samples % image name symbol name
22584 28.5982 postgres ExecInterpExpr
11950 15.1323 postgres slot_getmissingattrs
5471 6.9279 postgres AllocSetAlloc
3018 3.8217 postgres TupleDescInitEntry
2042 2.5858 postgres MemoryContextAllocZeroAligned
1807 2.2882 postgres SearchCatCache1
1638 2.0742 postgres expression_tree_walkerThat's very different from what we see on the master branch. The time
spent in slot_getmissingattrs is perhaps not unexpected, but I don't
(at least yet) understand why we're getting so much time spent in
ExecInterpExpr, which doesn't show up at all when the same benchmark
is run on master.I'd guess that's because the physical tlist optimization is disabled. I
assume you'd see something similar on master if you dropped a column.
I put that to the test and there's still something fishy going on.
I created some benchmark scripts to help out a bit. Basically, they'll
allow you to choose how many columns you want in the test tables and
how many rows to put in them.
The following test is a 60 second single threaded pgbench test with
1000 columns, 400 rows, testing select sum(c10) from <table>
*** Benchmarking normal table...
tps = 1972.985639 (excluding connections establishing)
*** Benchmarking missing table...
tps = 433.779008 (excluding connections establishing)
*** Benchmarking dropped table...
tps = 3661.063986 (excluding connections establishing)
The dropped table had 1001 columns, but the script drops the first.
The missing table has all 1000 columns missing. In the normal table
all tuples have all attrs.
I imagine it should be possible to make the missing table case faster
than either of the others. The reason it's slower is down to
slot_getsomeattrs being called to get all 1000 attributes in all but
the 2nd call to the function... ? whereas only the 10th attr is
deformed in the Normal table case.
Here's some perf output for each case:
Normal:
15.88% postgres postgres [.] expression_tree_walker
7.82% postgres postgres [.] fix_expr_common
6.12% postgres [kernel.kallsyms] [k] __lock_text_start
5.45% postgres postgres [.] SearchCatCache3
3.98% postgres postgres [.] TupleDescInitEntry
Missing:
47.80% postgres postgres [.] ExecInterpExpr
25.65% postgres postgres [.] slot_getmissingattrs
6.86% postgres postgres [.] build_tlist_index
4.43% postgres libc-2.17.so [.] __memset_sse2
2.57% postgres postgres [.] expression_tree_walker
Dropped:
19.59% postgres postgres [.] slot_deform_tuple
6.63% postgres postgres [.] hash_search_with_hash_value
6.55% postgres postgres [.] heap_getnext
5.35% postgres postgres [.] ExecInterpExpr
4.79% postgres postgres [.] heap_page_prune_opt
/me studies the code for a bit...
Okay, it looks like the patch should disable physical tlists when
there's a missing column the same way as we do for dropped columns.
Patch attached.
TPS looks much better now;
*** Benchmarking missing table...
tps = 5666.117627 (excluding connections establishing)
everybody's going to want missing columns in their tables now...
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
fast_default_disable_physical_tlists.patchapplication/octet-stream; name=fast_default_disable_physical_tlists.patchDownload
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b799e249db8..2642cf20c44 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1524,9 +1524,9 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
Form_pg_attribute att_tup = TupleDescAttr(relation->rd_att,
attrno - 1);
- if (att_tup->attisdropped)
+ if (att_tup->attisdropped || att_tup->atthasmissing)
{
- /* found a dropped col, so punt */
+ /* found a dropped or missing col, so punt */
tlist = NIL;
break;
}
On 6 March 2018 at 22:40, David Rowley <david.rowley@2ndquadrant.com> wrote:
Okay, it looks like the patch should disable physical tlists when
there's a missing column the same way as we do for dropped columns.
Patch attached.
Please disregard the previous patch in favour of the attached.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
fast_default_disable_physical_tlists_v2.patchapplication/octet-stream; name=fast_default_disable_physical_tlists_v2.patchDownload
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index b799e249db8..0f6c121c0db 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1484,12 +1484,12 @@ relation_excluded_by_constraints(PlannerInfo *root,
* in order. The executor can special-case such tlists to avoid a projection
* step at runtime, so we use such tlists preferentially for scan nodes.
*
- * Exception: if there are any dropped columns, we punt and return NIL.
- * Ideally we would like to handle the dropped-column case too. However this
- * creates problems for ExecTypeFromTL, which may be asked to build a tupdesc
- * for a tlist that includes vars of no-longer-existent types. In theory we
- * could dig out the required info from the pg_attribute entries of the
- * relation, but that data is not readily available to ExecTypeFromTL.
+ * Exception: if there are any dropped or missing columns, we punt and return
+ * NIL. Ideally we would like to handle the dropped-column case too. However
+ * this creates problems for ExecTypeFromTL, which may be asked to build a
+ * tupdesc for a tlist that includes vars of no-longer-existent types. In
+ * theory we could dig out the required info from the pg_attribute entries of
+ * the relation, but that data is not readily available to ExecTypeFromTL.
* For now, we don't apply the physical-tlist optimization when there are
* dropped cols.
*
@@ -1524,9 +1524,9 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
Form_pg_attribute att_tup = TupleDescAttr(relation->rd_att,
attrno - 1);
- if (att_tup->attisdropped)
+ if (att_tup->attisdropped || att_tup->atthasmissing)
{
- /* found a dropped col, so punt */
+ /* found a dropped or missing col, so punt */
tlist = NIL;
break;
}
On Tue, Mar 6, 2018 at 8:15 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:
On 6 March 2018 at 22:40, David Rowley <david.rowley@2ndquadrant.com> wrote:
Okay, it looks like the patch should disable physical tlists when
there's a missing column the same way as we do for dropped columns.
Patch attached.Please disregard the previous patch in favour of the attached.
OK, nice work, thanks. We're making good progress. Two of the cases I
was testing have gone from worse than master to better than master.
Here are the results I got, all against Tomas' 100-col 64-row table.
using pgbench -T 100:
select sum(c1000) from t;
fastdef tps = 4724.988619
master tps = 1590.843085
============
select c1000 from t;
fastdef tps = 5093.667203
master tps = 2437.613368
============
select sum(c1000) from (select c1000 from t offset 0) x;
fastdef tps = 3315.900091
master tps = 2067.574581
============
select * from t;
fastdef tps = 107.145811
master tps = 150.207957
============
select sum(c1), count(c500), avg(c1000) from t;
fastdef tps = 2304.636410
master tps = 1409.791975
============
select sum(c10) from t;
fastdef tps = 4332.625917
master tps = 2208.757119
"select * from t" used to be about a wash, but with this patch it's
got worse. The last two queries were worse and are now better, so
that's a win. I'm going to do a test to see if I can find the
break-even point between the second query and the fourth. If it turns
out to be at quite a large number of columns selected then I think it
might be something we can live with.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 8 March 2018 at 18:40, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
select * from t;
fastdef tps = 107.145811
master tps = 150.207957"select * from t" used to be about a wash, but with this patch it's
got worse. The last two queries were worse and are now better, so
that's a win.
How does it compare to master if you drop a column out the table?
Physical tlists will be disabled in that case too. I imagine the
performance of master will drop much lower than the all columns
missing case.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On 9 March 2018 at 02:11, David Rowley <david.rowley@2ndquadrant.com> wrote:
On 8 March 2018 at 18:40, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
select * from t;
fastdef tps = 107.145811
master tps = 150.207957"select * from t" used to be about a wash, but with this patch it's
got worse. The last two queries were worse and are now better, so
that's a win.How does it compare to master if you drop a column out the table?
Physical tlists will be disabled in that case too. I imagine the
performance of master will drop much lower than the all columns
missing case.
I decided to test this for myself, and the missing version is still
slightly slower than the dropped column version, but not by much. I'm
not personally concerned about this.
The following results are with 1000 column tables with 64 rows each.
*** Benchmarking normal table...
tps = 232.794734 (excluding connections establishing)
*** Benchmarking missing table...
tps = 202.827754 (excluding connections establishing)
*** Benchmarking dropped table...
tps = 217.339255 (excluding connections establishing)
There's nothing particularly interesting in the profiles for the
missing and normal case either:
-- Missing case for select * from missing;
12.87% postgres postgres [.] AllocSetAlloc
10.09% postgres libc-2.17.so [.] __strlen_sse2_pminub
6.35% postgres postgres [.] pq_sendcountedtext
5.97% postgres postgres [.] enlargeStringInfo
5.47% postgres postgres [.] ExecInterpExpr
4.45% postgres libc-2.17.so [.] __memcpy_ssse3_back
4.39% postgres postgres [.] palloc
4.31% postgres postgres [.] printtup
3.81% postgres postgres [.] pg_ltoa
3.72% postgres [kernel.kallsyms] [k] __lock_text_start
3.38% postgres postgres [.] FunctionCall1Coll
3.11% postgres postgres [.] int4out
2.97% postgres postgres [.] appendBinaryStringInfoNT
2.49% postgres postgres [.] slot_getmissingattrs
1.66% postgres postgres [.] pg_server_to_any
0.99% postgres postgres [.] MemoryContextAllocZeroAligned
0.94% postgres postgres [.] expression_tree_walker
0.93% postgres postgres [.] lappend
0.89% postgres [kernel.kallsyms] [k] __do_page_fault
0.72% postgres [kernel.kallsyms] [k] clear_page_c_e
0.72% postgres postgres [.] SendRowDescriptionMessage
0.71% postgres postgres [.] expandRelAttrs
0.64% postgres [kernel.kallsyms] [k] get_page_from_freelist
0.64% postgres [kernel.kallsyms] [k] copy_user_enhanced_fast_string
0.62% postgres postgres [.] pg_server_to_client
0.59% postgres libc-2.17.so [.] __memset_sse2
0.58% postgres postgres [.] set_pathtarget_cost_width
0.56% postgres postgres [.] OutputFunctionCall
0.56% postgres postgres [.] memcpy@plt
0.54% postgres postgres [.] makeTargetEntry
0.50% postgres postgres [.] SearchCatCache3
-- Normal case for select * from normal;
12.57% postgres postgres [.] AllocSetAlloc
10.50% postgres libc-2.17.so [.] __strlen_sse2_pminub
7.65% postgres postgres [.] slot_deform_tuple
6.52% postgres postgres [.] pq_sendcountedtext
6.03% postgres postgres [.] enlargeStringInfo
4.51% postgres postgres [.] printtup
4.35% postgres postgres [.] palloc
4.34% postgres libc-2.17.so [.] __memcpy_ssse3_back
3.90% postgres postgres [.] pg_ltoa
3.49% postgres [kernel.kallsyms] [k] __lock_text_start
3.35% postgres postgres [.] int4out
3.31% postgres postgres [.] FunctionCall1Coll
2.82% postgres postgres [.] appendBinaryStringInfoNT
1.68% postgres postgres [.] pg_server_to_any
1.30% postgres postgres [.] SearchCatCache3
1.01% postgres postgres [.] SendRowDescriptionMessage
0.89% postgres postgres [.] lappend
0.88% postgres postgres [.] MemoryContextAllocZeroAligned
0.79% postgres postgres [.] expression_tree_walker
0.67% postgres postgres [.] SearchCatCache1
0.67% postgres postgres [.] expandRelAttrs
0.64% postgres [kernel.kallsyms] [k] copy_user_enhanced_fast_string
0.60% postgres postgres [.] pg_server_to_client
0.57% postgres [kernel.kallsyms] [k] __do_page_fault
0.56% postgres postgres [.] OutputFunctionCall
0.53% postgres postgres [.] memcpy@plt
0.53% postgres postgres [.] makeTargetEntry
0.51% postgres [kernel.kallsyms] [k] get_page_from_freelist
The difference appears to be in ExecInterpExpr, although, that only
accounts for about 5.5%, and we're missing 15% in performance.
Going by the commitfest app, the patch still does appear to be waiting
on Author. Never-the-less, I've made another pass over it and found a
few mistakes and a couple of ways to improve things:
1. Should <structname> not be <structfield>?
<entry>
This column has a value which is used where the column is entirely
missing from the row, as happens when a column is added after the
row is created. The actual value used is stored in the
<structname>attmissingval</structname> column.
</entry>
Also additional surplus space before "column".
The same mixup appears in:
<structname>atthasmissing</structname> is true. If there is no value
2. Instead of slot_getmissingattrs() having a memset() to set all the
missing attributes to NULL before resetting them to the missing value,
if one is found. Would it not be better to just have the missing array
store both the NULLs and the DEFAULTs together?
With that, the code could become:
static void
slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
{
AttrMissing *attrmiss;
int attnum;
if (slot->tts_tupleDescriptor->constr &&
slot->tts_tupleDescriptor->constr->missing)
attrmiss = slot->tts_tupleDescriptor->constr->missing;
else
return;
for (attnum = startAttNum; attnum < lastAttNum; attnum++)
{
slot->tts_values[attnum] = attrmiss[attnum].ammissing;
slot->tts_isnull[attnum] = false;
}
}
You may even be able to just:
Assert(slot->tts_tupleDescriptor->constr);
Assert(slot->tts_tupleDescriptor->constr->missing);
instead of the run-time check.
This would also simplify expand_tuple() and getmissingattr()
I think this change makes sense as slot_getmissingattrs() is getting
NULLs and the missing values, not just the missing values as the name
indicates. The name would become more fitting if you class a NULL as a
missing value.
3. Not sure what amnum means. Is it meant to be adnum?
* the source (i.e. where its amnum is > sourceNatts). We can ignore
4. In StoreAttrDefault(), is there any point in creating the executor
state when add_column_mode == false?
It's probably also worth commenting what add_column_mode does. If any
caller mistakenly passes it as 'true' then you'll overwrite the
original missing value which would appear to update tuples that
already exist in the table.
It also looks like you could move the bulk of the new code in that
function, including the new variables into where you're modifying the
pg_attribute tuple.
This function also incorrectly sets atthasmissing to true even when
the DEFAULT expression evaluates to NULL. This seems to contradict
what you're doing in RelationBuildTupleDesc(), which skips building
the missing array unless it sees a non-null attmissingval.
This will cause getmissingattr() to Assert fail with the following case:
drop table if exists t1;
create table t1 (a int);
insert into t1 values(1);
alter table t1 add column b text default 'test' || null;
alter table t1 add column c text null;
select attname,atthasmissing,attmissingval from pg_attribute where
attrelid = 't1'::regclass and attnum>0;
select b from t1;
TRAP: FailedAssertion("!(tupleDesc->constr->missing)", File:
"src/backend/access/common/heaptuple.c", Line: 98)
Probably another good reason to do the refactor work mentioned in #2.
It might be worth adding a regression test to ensure atthasmissing
remains false for a column where the default expression evaluated to
NULL;
5. It's not really required, but in AddRelationNewConstraints() you
could get away with only performing the volatile function check if
missingMode is true. That would save calling
contain_volatile_functions() needlessly.
6. The comment above RelationClearMissing() should probably mention
that the caller must hold an AccessExclusiveLock on rel. There is also
a surplus newline between the header comment and the function.
7. Can you pg_ident the patch? There's quite a few places where the
whitespace is messed up, and also quite a few places where you've
changed lines so they're longer than 80 chars.
8. In RelationBuildTupleDesc() it looks like attnum can be declared in
the scope of the while loop.
An updated patch should probably also included the patch I sent to
disable the physical tlists when there are missing values mentioned in
the TupleDesc. Although I admit to not having properly updated the
comments even in my v2 attempt.
* NIL. Ideally we would like to handle the dropped-column case too. However
should read:
* NIL. Ideally, we would like to handle these cases too. However
Thanks again for working on this feature. I hope we can get this into PG11.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
On Mon, Mar 12, 2018 at 1:29 AM, David Rowley
<david.rowley@2ndquadrant.com> wrote:
On 9 March 2018 at 02:11, David Rowley <david.rowley@2ndquadrant.com> wrote:
On 8 March 2018 at 18:40, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote:
select * from t;
fastdef tps = 107.145811
master tps = 150.207957"select * from t" used to be about a wash, but with this patch it's
got worse. The last two queries were worse and are now better, so
that's a win.How does it compare to master if you drop a column out the table?
Physical tlists will be disabled in that case too. I imagine the
performance of master will drop much lower than the all columns
missing case.I decided to test this for myself, and the missing version is still
slightly slower than the dropped column version, but not by much. I'm
not personally concerned about this.The following results are with 1000 column tables with 64 rows each.
I've done some more extensive benchmarking now. Here are some fairly
typical results from pgbench runs done on standard scale 100 pgbench
data:
[andrew@foo tests]$ PATH=$HOME/pg_fast_def/root/HEAD/inst/bin:$PATH
pgbench -S -c 10 -j 5 -T 60 test
starting vacuum...end.
transaction type: <builtin: select only>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 60 s
number of transactions actually processed: 2235601
latency average = 0.268 ms
tps = 37256.886332 (including connections establishing)
tps = 37258.562925 (excluding connections establishing)
[andrew@foo tests]$ PATH=$HOME/pg_head/root/HEAD/inst/bin:$PATH
pgbench -S -c 10 -j 5 -T 60 test
starting vacuum...end.
transaction type: <builtin: select only>
scaling factor: 100
query mode: simple
number of clients: 10
number of threads: 5
duration: 60 s
number of transactions actually processed: 2230085
latency average = 0.269 ms
tps = 37164.696271 (including connections establishing)
tps = 37166.647971 (excluding connections establishing)
So generally the patched code and master are pretty much on a par.
I have also done some testing on cases meant to stress-test the
feature a bit - two 1000 column, all columns having defaults, one with
a dropped column. For the fast_default case I then also copied the
tables (and again dropped a column) so that the data files and table
definitions would match fairly closely what was being tested in the
master branch. The scripts in the attached tests.tgz. The test
platform is an Amazon r4.2xlarge instance running RHEL7.
There are two sets of results attached, one for 64 row tables and one
for 50k row tables.
The 50k row results are fairly unambiguous, the patched code performs
as well as or better (in some cases spectacularly better) than master.
In a few cases the patched code performs slightly worse than master in
the last (fdnmiss) case with the copied tables.
Going by the commitfest app, the patch still does appear to be waiting
on Author. Never-the-less, I've made another pass over it and found a
few mistakes and a couple of ways to improve things:
working on these. Should have a new patch tomorrow.
Thanks again for working on this feature. I hope we can get this into PG11.
Thanks for you help. I hope so too.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
Going by the commitfest app, the patch still does appear to be waiting
on Author. Never-the-less, I've made another pass over it and found a
few mistakes and a couple of ways to improve things:working on these. Should have a new patch tomorrow.
Here is a patch that attends to most of these, except that I haven't
re-indented it.
Your proposed changes to slot_getmissingattrs() wouldn't work, but I
have rewritten it in a way that avoids the double work you disliked.
I'll rerun the benchmark tests I posted upthread and let you know the results.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v15.patchapplication/octet-stream; name=fast_default-v15.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a0e6d7062b..5ef1ddc069 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structfield>attmissingval</structfield> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,19 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>anyarray</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a one element array containing the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structfield>atthasmissing</structfield> is true. If there is no value
+ the column is null.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..edf459ae6b 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1184,24 +1184,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..4b138b16d9 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -58,6 +58,7 @@
#include "postgres.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -76,6 +77,73 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->missing);
+
+ attrmiss = tupleDesc->constr->missing + (attnum - 1);
+
+ if (attrmiss->ammissingPresent)
+ {
+ *isnull = false;
+ return attrmiss->ammissing;
+ }
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss = NULL;
+ int missattnum;
+
+ if (slot->tts_tupleDescriptor->constr)
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+
+
+ if(!attrmiss)
+ {
+ /* no missing values array at all, so just fill everything in as NULL */
+ memset(slot->tts_values + startAttNum, 0,
+ (lastAttNum - startAttNum) * sizeof(Datum));
+ memset(slot->tts_isnull + startAttNum, 1,
+ (lastAttNum - startAttNum) * sizeof(bool));
+ }
+ else
+ {
+ /* if there is a missing values array we must process them one by one */
+ for (missattnum = lastAttNum - 1;
+ missattnum >= startAttNum;
+ missattnum--)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] =
+ ! attrmiss[missattnum].ammissingPresent;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +200,134 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Per-attribute helper for heap_fill_tuple and other routines building tuples.
+ *
+ * Fill in either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+
+
+
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +368,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +393,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have
+ * missing values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if(tupleDesc && TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ return false;
+ else
+ return true;
+ }
if (attnum > 0)
{
@@ -649,6 +759,275 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple which has less attributes than required. For each attribute
+ * not present in the sourceTuple, if there is a missing value that will be
+ * used. Otherwise the attribute will be set to NULL.
+ *
+ * The source tuple must have less attributes than the required number.
+ *
+ * Only one of targetHeapTuple and targetMinimalTuple may be supplied. The
+ * other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int firstmissingnum = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceNullLen;
+ int targetNullLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceNullLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source. We can ignore all the missing entries before that.
+ */
+ for (firstmissingnum = sourceNatts;
+ firstmissingnum < natts;
+ firstmissingnum++)
+ {
+ if (attrmiss[firstmissingnum].ammissingPresent)
+ break;
+ }
+
+ /*
+ * If there are no more missing values everything else must be
+ * NULL
+ */
+ if (firstmissingnum >= natts)
+ {
+ hasNulls = true;
+ }
+ else
+ {
+
+ /*
+ * Now walk the missing attributes. If there is a missing value
+ * make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = firstmissingnum;
+ attnum < natts;
+ attnum++)
+ {
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum);
+
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+ }
+ else
+ {
+ /* no missing value, so it must be null */
+ hasNulls = true;
+ }
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetNullLen = BITMAPLEN(natts);
+ len += targetNullLen;
+ }
+ else
+ targetNullLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetNullLen > 0)
+ {
+ if (sourceNullLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceNullLen);
+ nullBits += sourceNullLen - 1;
+ }
+ else
+ {
+ sourceNullLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceNullLen);
+
+ nullBits += sourceNullLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ targetData += sourceDataLen;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts ; attnum < natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[attnum].ammissing,
+ false);
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1391,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as nulls or missing values as appropriate.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum +1, &isnull[attnum]);
}
/*
@@ -1192,8 +1568,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1638,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1683,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1711,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1734,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..2b7d53ae2f 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +178,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if(constr->missing)
+ {
+ cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissingPresent)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->missing)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissingPresent
+ && !TupleDescAttr(tupdesc, i)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +500,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
+
+ if (missval1->ammissingPresent != missval2->ammissingPresent)
+ return false;
+ if (missval1->ammissingPresent)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ }
+ }
+ else if (constr2->missing)
+ return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +638,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,6 +852,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 0648539796..83000575ce 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4588,7 +4588,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..4a056cf4d1 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1921,14 +1927,91 @@ heap_drop_with_catalog(Oid relid)
}
+/*
+ * RelationClearMissing
+ *
+ * set atthasmissing and attmissingval to false/null for all attributes
+ * where they are currently set. This can be safely and usefully done if
+ * the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
+ * are no rows left with less than a full complement of attributes.
+ *
+ * The caller must have an AccessExclusive lock on the relation.
+ */
+
+void
+RelationClearMissing(Relation rel)
+{
+ Relation attr_rel;
+ Oid relid = RelationGetRelid(rel);
+ int natts = RelationGetNumberOfAttributes(rel);
+ int attnum;
+ Datum repl_val[Natts_pg_attribute];
+ bool repl_null[Natts_pg_attribute];
+ bool repl_repl[Natts_pg_attribute];
+ Form_pg_attribute attrtuple;
+ HeapTuple tuple,
+ newtuple;
+
+ memset(repl_val, 0, sizeof(repl_val));
+ memset(repl_null, false, sizeof(repl_null));
+ memset(repl_repl, false, sizeof(repl_repl));
+
+ repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false);
+ repl_null[Anum_pg_attribute_attmissingval - 1] = true;
+
+ repl_repl[Anum_pg_attribute_atthasmissing - 1] = true;
+ repl_repl[Anum_pg_attribute_attmissingval - 1] = true;
+
+
+ /* Get a lock on pg_attribute */
+ attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ /* process each non-system attribute */
+ for (attnum = 1; attnum <= natts; attnum++)
+ {
+ tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+ if (!HeapTupleIsValid(tuple)) /* shouldn't happen */
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /* ignore any where atthasmissing is not true */
+ if (attrtuple->atthasmissing)
+ {
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attr_rel),
+ repl_val, repl_null, repl_repl);
+
+ CatalogTupleUpdate(attr_rel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+
+ ReleaseSysCache(tuple);
+ }
+ /*
+ * Our update of the pg_attribute rows will force a relcache rebuild, so
+ * there's nothing else to do here.
+ */
+ heap_close(attr_rel, RowExclusiveLock);
+}
+
/*
* Store a default expression for column attnum of relation rel.
*
* Returns the OID of the new pg_attrdef tuple.
+ *
+ * add_column_mode must be true if we are storing the default for a new
+ * attribute, and false if it's for an already existing attribute. The reason
+ * for this is that the missing value must never be updated after it is set,
+ * which can only be when a column is added to the table. Otherwise we would
+ * in effect be changing existing tuples.
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1939,9 +2022,19 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1996,7 +2089,50 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+
+ if (add_column_mode)
+ {
+ expr2 = expression_planner(expr2);
+ estate = CreateExecutorState();
+ exprState = ExecPrepareExpr(expr2, estate);
+ econtext = GetPerTupleExprContext(estate);
+
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ /* if the default evaluates to NULL, just store a NULL array */
+ missingval = (Datum) 0;
+ }
+ else
+ {
+ /* otherwise make a one-element array of the value */
+ missingval = PointerGetDatum(
+ construct_array(&missingval,
+ 1,
+ defAttStruct->atttypid,
+ defAttStruct->attlen,
+ defAttStruct->attbyval,
+ defAttStruct->attalign));
+ }
+
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = ! missingIsNull;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2028,6 +2164,14 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
return attrdefOid;
}
@@ -2180,7 +2324,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2296,7 +2440,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (colDef->missingMode && contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 431bc31969..0a2a4be74d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1668,7 +1669,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..2428b50dba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
@@ -1668,6 +1668,16 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
relation_close(newrel, NoLock);
}
+
+ /* if it's not a catalog table, clear any missing attribute settings */
+ if (!is_system_catalog)
+ {
+ Relation newrel;
+
+ newrel = heap_open(OIDOldHeap, NoLock);
+ RelationClearMissing(newrel);
+ relation_close(newrel, NoLock);
+ }
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index b1f87d056e..c9b8b81ad0 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2251,7 +2251,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic)
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index ff40683d06..c38da2e8ca 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 218224a156..25e160a826 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4681,7 +4682,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4784,7 +4785,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5403,6 +5404,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5446,6 +5448,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5456,6 +5459,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5501,6 +5513,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5536,16 +5551,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6021,6 +6041,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8107,8 +8128,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9514,7 +9535,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index bf3cd3a454..b92aa3baff 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2397,7 +2397,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 771b7e3945..e1c1fb741a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2483,7 +2483,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 91ba939bdc..aa5c7cbe0e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf93..025d3e379c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a8ae37ebc8..7012a2e8b3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index a9a09afd2b..59b931afd0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->prokind != PROKIND_FUNCTION ||
funcform->proretset ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index bd3a0c4a0a..5348c77b23 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1493,12 +1493,12 @@ relation_excluded_by_constraints(PlannerInfo *root,
* in order. The executor can special-case such tlists to avoid a projection
* step at runtime, so we use such tlists preferentially for scan nodes.
*
- * Exception: if there are any dropped columns, we punt and return NIL.
- * Ideally we would like to handle the dropped-column case too. However this
- * creates problems for ExecTypeFromTL, which may be asked to build a tupdesc
- * for a tlist that includes vars of no-longer-existent types. In theory we
- * could dig out the required info from the pg_attribute entries of the
- * relation, but that data is not readily available to ExecTypeFromTL.
+ * Exception: if there are any dropped or missing columns, we punt and return
+ * NIL. Ideally we would like to handle these cases too. However
+ * this creates problems for ExecTypeFromTL, which may be asked to build a
+ * tupdesc for a tlist that includes vars of no-longer-existent types. In
+ * theory we could dig out the required info from the pg_attribute entries of
+ * the relation, but that data is not readily available to ExecTypeFromTL.
* For now, we don't apply the physical-tlist optimization when there are
* dropped cols.
*
@@ -1533,9 +1533,9 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
Form_pg_attribute att_tup = TupleDescAttr(relation->rd_att,
attrno - 1);
- if (att_tup->attisdropped)
+ if (att_tup->attisdropped || att_tup->atthasmissing)
{
- /* found a dropped col, so punt */
+ /* found a dropped or missing col, so punt */
tlist = NIL;
break;
}
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..361bde4261 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index b58ee3c387..1f81ad64bd 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1240,7 +1240,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1408,7 +1408,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1642,7 +1642,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db26bd..6ca66ea82b 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -36,6 +36,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,6 +495,7 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
/* copy some fields from pg_class row to rd_att */
@@ -538,15 +541,17 @@ RelationBuildTupleDesc(Relation relation)
while (HeapTupleIsValid(pg_attribute_tuple = systable_getnext(pg_attribute_scan)))
{
Form_pg_attribute attp;
+ int attnum;
attp = (Form_pg_attribute) GETSTRUCT(pg_attribute_tuple);
- if (attp->attnum <= 0 ||
- attp->attnum > relation->rd_rel->relnatts)
+ attnum = attp->attnum;
+ if (attnum <= 0 || attnum > relation->rd_rel->relnatts)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
- memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
+
+ memcpy(TupleDescAttr(relation->rd_att, attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +559,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +567,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
@@ -605,7 +664,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ attrmiss || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +680,10 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+ constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4056,10 +4119,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4398,7 +4457,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4693,7 +4752,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4755,7 +4814,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index c0076bfce3..d5580deddc 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..6d8051b850 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -25,6 +25,8 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing *MissingPtr;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,6 +40,7 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ MissingPtr missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
bool has_not_null;
diff --git a/src/include/access/tupdesc_details.h b/src/include/access/tupdesc_details.h
new file mode 100644
index 0000000000..c5d9605974
--- /dev/null
+++ b/src/include/access/tupdesc_details.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * tupdesc_details.h
+ * POSTGRES tuple descriptor definitions we can't include everywhere
+ *
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/tupdesc_details.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TUPDESC_DETAILS_H
+#define TUPDESC_DETAILS_H
+
+/*
+ * Structure used to represent value to be used when the attribute is not
+ * present at all in a tuple, i.e. when the column was created after the tuple
+ */
+
+typedef struct attrMissing
+{
+ bool ammissingPresent; /* true if non-NULL missing value exists */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
+#endif /* TUPDESC_DETAILS_H */
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..69adbc5d1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -102,8 +103,11 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_local,
bool is_internal);
+extern void RelationClearMissing(Relation rel);
+
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..280e36c5d2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +200,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +215,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 01cf59e7a3..3bd4e3dae0 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..9e74068a0a
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..c4bf9d61e4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing
+test: identity partition_join partition_prune reloptions hash_part indexing fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..6a3f9b367e
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On Tue, Mar 13, 2018 at 10:58 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Going by the commitfest app, the patch still does appear to be waiting
on Author. Never-the-less, I've made another pass over it and found a
few mistakes and a couple of ways to improve things:working on these. Should have a new patch tomorrow.
Here is a patch that attends to most of these, except that I haven't
re-indented it.Your proposed changes to slot_getmissingattrs() wouldn't work, but I
have rewritten it in a way that avoids the double work you disliked.I'll rerun the benchmark tests I posted upthread and let you know the results.
Here are the benchmark results from the v15 patch. Fairly similar to
previous results. I'm going to run some profiling again to see if I
can identify any glaring hotspots. I do suspect that the "physical
tlist" optimization sometimes turns out not to be one. It seems
perverse to be able to improve a query's performance by dropping a
column.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 14 March 2018 at 11:36, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
Here are the benchmark results from the v15 patch. Fairly similar to
previous results. I'm going to run some profiling again to see if I
can identify any glaring hotspots. I do suspect that the "physical
tlist" optimization sometimes turns out not to be one. It seems
perverse to be able to improve a query's performance by dropping a
column.
Can you explain what "fdnmiss" is that appears in the results?
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Hi,
On 2018-03-14 09:06:54 +1030, Andrew Dunstan wrote:
I do suspect that the "physical tlist" optimization sometimes turns
out not to be one. It seems perverse to be able to improve a query's
performance by dropping a column.
Yea, it's definitely not always a boon. Especially w/ the newer v10+
project code. I suspect we should largely get rid of it in v12, which
presumably will see a storage layer abstraction...
It'll be even more worthwhile to get rid of it if we manage to avoid
deforming columns that aren't needed but are lower than the currently
required columns. I still think we should build bitmasks of required
columns during planning.
Greetings,
Andres Freund
On Mar 14, 2018, at 10:58 AM, David Rowley <david.rowley@2ndquadrant.com> wrote:
On 14 March 2018 at 11:36, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Here are the benchmark results from the v15 patch. Fairly similar to
previous results. I'm going to run some profiling again to see if I
can identify any glaring hotspots. I do suspect that the "physical
tlist" optimization sometimes turns out not to be one. It seems
perverse to be able to improve a query's performance by dropping a
column.Can you explain what "fdnmiss" is that appears in the results?
It’s the patched code run against a materialized version of the table, i.e. one with no missing attributes.
Cheers
Andrew
On Tue, Mar 13, 2018 at 10:58 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Tue, Mar 13, 2018 at 2:40 PM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:Going by the commitfest app, the patch still does appear to be waiting
on Author. Never-the-less, I've made another pass over it and found a
few mistakes and a couple of ways to improve things:working on these. Should have a new patch tomorrow.
Here is a patch that attends to most of these, except that I haven't
re-indented it.Your proposed changes to slot_getmissingattrs() wouldn't work, but I
have rewritten it in a way that avoids the double work you disliked.
rebased and mostly indented patch version attached. It's actually
moderately difficult to produce a nicely indented patch that is
against non-pgindented code. We should look at that a bit. Another
item for my TODO list.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
fast_default-v16.patchapplication/octet-stream; name=fast_default-v16.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 30e6741305..8e03bc22d9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structfield>attmissingval</structfield> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,19 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>anyarray</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a one element array containing the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structfield>atthasmissing</structfield> is true. If there is no value
+ the column is null.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..edf459ae6b 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1184,24 +1184,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..621ad0c5e7 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -58,6 +58,7 @@
#include "postgres.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -76,6 +77,73 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->missing);
+
+ attrmiss = tupleDesc->constr->missing + (attnum - 1);
+
+ if (attrmiss->ammissingPresent)
+ {
+ *isnull = false;
+ return attrmiss->ammissing;
+ }
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss = NULL;
+ int missattnum;
+
+ if (slot->tts_tupleDescriptor->constr)
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+
+
+ if (!attrmiss)
+ {
+ /* no missing values array at all, so just fill everything in as NULL */
+ memset(slot->tts_values + startAttNum, 0,
+ (lastAttNum - startAttNum) * sizeof(Datum));
+ memset(slot->tts_isnull + startAttNum, 1,
+ (lastAttNum - startAttNum) * sizeof(bool));
+ }
+ else
+ {
+ /* if there is a missing values array we must process them one by one */
+ for (missattnum = lastAttNum - 1;
+ missattnum >= startAttNum;
+ missattnum--)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] =
+ !attrmiss[missattnum].ammissingPresent;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +200,134 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Per-attribute helper for heap_fill_tuple and other routines building tuples.
+ *
+ * Fill in either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+
+
+
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +368,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +393,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have missing
+ * values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc && TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ return false;
+ else
+ return true;
+ }
if (attnum > 0)
{
@@ -649,6 +759,274 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple which has less attributes than required. For each attribute
+ * not present in the sourceTuple, if there is a missing value that will be
+ * used. Otherwise the attribute will be set to NULL.
+ *
+ * The source tuple must have less attributes than the required number.
+ *
+ * Only one of targetHeapTuple and targetMinimalTuple may be supplied. The
+ * other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int firstmissingnum = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceNullLen;
+ int targetNullLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceNullLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source. We can ignore all the missing entries before that.
+ */
+ for (firstmissingnum = sourceNatts;
+ firstmissingnum < natts;
+ firstmissingnum++)
+ {
+ if (attrmiss[firstmissingnum].ammissingPresent)
+ break;
+ }
+
+ /*
+ * If there are no more missing values everything else must be NULL
+ */
+ if (firstmissingnum >= natts)
+ {
+ hasNulls = true;
+ }
+ else
+ {
+
+ /*
+ * Now walk the missing attributes. If there is a missing value
+ * make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = firstmissingnum;
+ attnum < natts;
+ attnum++)
+ {
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum);
+
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+ }
+ else
+ {
+ /* no missing value, so it must be null */
+ hasNulls = true;
+ }
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetNullLen = BITMAPLEN(natts);
+ len += targetNullLen;
+ }
+ else
+ targetNullLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetNullLen > 0)
+ {
+ if (sourceNullLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceNullLen);
+ nullBits += sourceNullLen - 1;
+ }
+ else
+ {
+ sourceNullLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceNullLen);
+
+ nullBits += sourceNullLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ targetData += sourceDataLen;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts; attnum < natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[attnum].ammissing,
+ false);
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1390,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as nulls or missing values as appropriate.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum + 1, &isnull[attnum]);
}
/*
@@ -1192,8 +1567,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1637,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1682,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1710,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1733,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..5439360722 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +178,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if (constr->missing)
+ {
+ cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissingPresent)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->missing)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissingPresent
+ && !TupleDescAttr(tupdesc, i)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +500,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
+
+ if (missval1->ammissingPresent != missval2->ammissingPresent)
+ return false;
+ if (missval1->ammissingPresent)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ }
+ }
+ else if (constr2->missing)
+ return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +638,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,6 +852,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 0648539796..83000575ce 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4588,7 +4588,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3d80ff9e5b..e672854539 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1920,14 +1926,92 @@ heap_drop_with_catalog(Oid relid)
}
+/*
+ * RelationClearMissing
+ *
+ * set atthasmissing and attmissingval to false/null for all attributes
+ * where they are currently set. This can be safely and usefully done if
+ * the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
+ * are no rows left with less than a full complement of attributes.
+ *
+ * The caller must have an AccessExclusive lock on the relation.
+ */
+
+void
+RelationClearMissing(Relation rel)
+{
+ Relation attr_rel;
+ Oid relid = RelationGetRelid(rel);
+ int natts = RelationGetNumberOfAttributes(rel);
+ int attnum;
+ Datum repl_val[Natts_pg_attribute];
+ bool repl_null[Natts_pg_attribute];
+ bool repl_repl[Natts_pg_attribute];
+ Form_pg_attribute attrtuple;
+ HeapTuple tuple,
+ newtuple;
+
+ memset(repl_val, 0, sizeof(repl_val));
+ memset(repl_null, false, sizeof(repl_null));
+ memset(repl_repl, false, sizeof(repl_repl));
+
+ repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false);
+ repl_null[Anum_pg_attribute_attmissingval - 1] = true;
+
+ repl_repl[Anum_pg_attribute_atthasmissing - 1] = true;
+ repl_repl[Anum_pg_attribute_attmissingval - 1] = true;
+
+
+ /* Get a lock on pg_attribute */
+ attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ /* process each non-system attribute */
+ for (attnum = 1; attnum <= natts; attnum++)
+ {
+ tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+ if (!HeapTupleIsValid(tuple)) /* shouldn't happen */
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /* ignore any where atthasmissing is not true */
+ if (attrtuple->atthasmissing)
+ {
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attr_rel),
+ repl_val, repl_null, repl_repl);
+
+ CatalogTupleUpdate(attr_rel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+
+ ReleaseSysCache(tuple);
+ }
+
+ /*
+ * Our update of the pg_attribute rows will force a relcache rebuild, so
+ * there's nothing else to do here.
+ */
+ heap_close(attr_rel, RowExclusiveLock);
+}
+
/*
* Store a default expression for column attnum of relation rel.
*
* Returns the OID of the new pg_attrdef tuple.
+ *
+ * add_column_mode must be true if we are storing the default for a new
+ * attribute, and false if it's for an already existing attribute. The reason
+ * for this is that the missing value must never be updated after it is set,
+ * which can only be when a column is added to the table. Otherwise we would
+ * in effect be changing existing tuples.
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1938,9 +2022,19 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -1995,7 +2089,50 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+
+ if (add_column_mode)
+ {
+ expr2 = expression_planner(expr2);
+ estate = CreateExecutorState();
+ exprState = ExecPrepareExpr(expr2, estate);
+ econtext = GetPerTupleExprContext(estate);
+
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ /* if the default evaluates to NULL, just store a NULL array */
+ missingval = (Datum) 0;
+ }
+ else
+ {
+ /* otherwise make a one-element array of the value */
+ missingval = PointerGetDatum(
+ construct_array(&missingval,
+ 1,
+ defAttStruct->atttypid,
+ defAttStruct->attlen,
+ defAttStruct->attbyval,
+ defAttStruct->attalign));
+ }
+
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = !missingIsNull;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2027,6 +2164,14 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
return attrdefOid;
}
@@ -2179,7 +2324,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2295,7 +2440,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (colDef->missingMode && contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9e2dd0e729..4e9b0d9194 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1652,7 +1653,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 5d481dd50d..2428b50dba 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
@@ -1668,6 +1668,16 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
relation_close(newrel, NoLock);
}
+
+ /* if it's not a catalog table, clear any missing attribute settings */
+ if (!is_system_catalog)
+ {
+ Relation newrel;
+
+ newrel = heap_open(OIDOldHeap, NoLock);
+ RelationClearMissing(newrel);
+ relation_close(newrel, NoLock);
+ }
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 86fa8c0dd7..c46493dd88 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2252,7 +2252,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic, DestReceiver
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index ff40683d06..c38da2e8ca 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 218224a156..25e160a826 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -713,6 +713,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4681,7 +4682,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4784,7 +4785,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5403,6 +5404,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5446,6 +5448,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5456,6 +5459,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5501,6 +5513,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5536,16 +5551,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6021,6 +6041,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8107,8 +8128,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9514,7 +9535,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index bf3cd3a454..b92aa3baff 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2397,7 +2397,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 771b7e3945..e1c1fb741a 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2483,7 +2483,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 91ba939bdc..aa5c7cbe0e 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2970,8 +2970,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf93..025d3e379c 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index a8ae37ebc8..7012a2e8b3 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -508,6 +508,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index a9a09afd2b..59b931afd0 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->prokind != PROKIND_FUNCTION ||
funcform->proretset ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5018,7 +5018,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->provolatile == PROVOLATILE_VOLATILE ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index bd3a0c4a0a..5348c77b23 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1493,12 +1493,12 @@ relation_excluded_by_constraints(PlannerInfo *root,
* in order. The executor can special-case such tlists to avoid a projection
* step at runtime, so we use such tlists preferentially for scan nodes.
*
- * Exception: if there are any dropped columns, we punt and return NIL.
- * Ideally we would like to handle the dropped-column case too. However this
- * creates problems for ExecTypeFromTL, which may be asked to build a tupdesc
- * for a tlist that includes vars of no-longer-existent types. In theory we
- * could dig out the required info from the pg_attribute entries of the
- * relation, but that data is not readily available to ExecTypeFromTL.
+ * Exception: if there are any dropped or missing columns, we punt and return
+ * NIL. Ideally we would like to handle these cases too. However
+ * this creates problems for ExecTypeFromTL, which may be asked to build a
+ * tupdesc for a tlist that includes vars of no-longer-existent types. In
+ * theory we could dig out the required info from the pg_attribute entries of
+ * the relation, but that data is not readily available to ExecTypeFromTL.
* For now, we don't apply the physical-tlist optimization when there are
* dropped cols.
*
@@ -1533,9 +1533,9 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
Form_pg_attribute att_tup = TupleDescAttr(relation->rd_att,
attrno - 1);
- if (att_tup->attisdropped)
+ if (att_tup->attisdropped || att_tup->atthasmissing)
{
- /* found a dropped col, so punt */
+ /* found a dropped or missing col, so punt */
tlist = NIL;
break;
}
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3..361bde4261 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12..7eb981a7ed 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 8faae1d069..30b7e62c09 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -205,7 +205,7 @@ static void ri_GenerateQual(StringInfo buf,
const char *rightop, Oid rightoptype);
static void ri_add_cast_to(StringInfo buf, Oid typid);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -308,7 +308,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -515,7 +515,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -725,7 +725,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -912,7 +912,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1072,7 +1072,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1286,7 +1286,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1502,7 +1502,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1677,7 +1677,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1733,7 +1733,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1764,7 +1764,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2058,7 +2058,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2915,7 +2915,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2930,7 +2931,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index b58ee3c387..1f81ad64bd 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1240,7 +1240,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1408,7 +1408,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1642,7 +1642,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db26bd..db6d8b0ceb 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -36,6 +36,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,6 +495,7 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
/* copy some fields from pg_class row to rd_att */
@@ -538,15 +541,17 @@ RelationBuildTupleDesc(Relation relation)
while (HeapTupleIsValid(pg_attribute_tuple = systable_getnext(pg_attribute_scan)))
{
Form_pg_attribute attp;
+ int attnum;
attp = (Form_pg_attribute) GETSTRUCT(pg_attribute_tuple);
- if (attp->attnum <= 0 ||
- attp->attnum > relation->rd_rel->relnatts)
+ attnum = attp->attnum;
+ if (attnum <= 0 || attnum > relation->rd_rel->relnatts)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
- memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
+
+ memcpy(TupleDescAttr(relation->rd_att, attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +559,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +567,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
@@ -605,7 +664,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ attrmiss || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +680,10 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+ constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4056,10 +4119,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4398,7 +4457,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4693,7 +4752,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4755,7 +4814,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index 25f4c34901..e72e4150de 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -199,7 +199,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 20f60392af..d5984b415e 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..fb1ad17012 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -795,7 +795,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -825,5 +825,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..708160f645 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -25,6 +25,8 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing *MissingPtr;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,6 +40,7 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ MissingPtr missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
bool has_not_null;
diff --git a/src/include/access/tupdesc_details.h b/src/include/access/tupdesc_details.h
new file mode 100644
index 0000000000..741e996b3c
--- /dev/null
+++ b/src/include/access/tupdesc_details.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * tupdesc_details.h
+ * POSTGRES tuple descriptor definitions we can't include everywhere
+ *
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/tupdesc_details.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TUPDESC_DETAILS_H
+#define TUPDESC_DETAILS_H
+
+/*
+ * Structure used to represent value to be used when the attribute is not
+ * present at all in a tuple, i.e. when the column was created after the tuple
+ */
+
+typedef struct attrMissing
+{
+ bool ammissingPresent; /* true if non-NULL missing value exists */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
+#endif /* TUPDESC_DETAILS_H */
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 9bdc63ceb5..69adbc5d1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -102,8 +103,11 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_local,
bool is_internal);
+extern void RelationClearMissing(Relation rel);
+
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..280e36c5d2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +200,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +215,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 7fc355acb8..f64a27cf1e 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -147,7 +147,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c6803081..008e859d4c 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 0000000000..9e74068a0a
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..c4bf9d61e4 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing
+test: identity partition_join partition_prune reloptions hash_part indexing fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..1aa0609263 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -187,3 +187,4 @@ test: hash_part
test: indexing
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 0000000000..6a3f9b367e
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 3/15/18 4:33 AM, Andrew Dunstan wrote:
rebased and mostly indented patch version attached. It's actually
moderately difficult to produce a nicely indented patch that is
against non-pgindented code. We should look at that a bit. Another
item for my TODO list.
It looks like this should be marked Needs Review so I have done so. If
that's not right please change it back or let me know and I will.
Regards,
--
-David
david@pgmasters.net
On 15 March 2018 at 21:33, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
rebased and mostly indented patch version attached.
Thanks. I've attached a version of this which applies, builds and
passes the regression tests on current master.
Some conflicts were caused by 325f2ec555 and there was a new call to
heap_attisnull which needed to be updated.
I'll look over this now.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
fast_default-v17.patchapplication/octet-stream; name=fast_default-v17.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 95a5b113b98..ebdb120a1a8 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1149,6 +1149,18 @@
</entry>
</row>
+ <row>
+ <entry><structfield>atthasmissing</structfield></entry>
+ <entry><type>bool</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a value which is used where the column is entirely
+ missing from the row, as happens when a column is added after the
+ row is created. The actual value used is stored in the
+ <structfield>attmissingval</structfield> column.
+ </entry>
+ </row>
+
<row>
<entry><structfield>attidentity</structfield></entry>
<entry><type>char</type></entry>
@@ -1229,6 +1241,19 @@
</entry>
</row>
+ <row>
+ <entry><structfield>attmissingval</structfield></entry>
+ <entry><type>anyarray</type></entry>
+ <entry></entry>
+ <entry>
+ This column has a one element array containing the value used when the
+ column is entirely missing from the row, as happens when the column is
+ added after the row is created. The value is only used when
+ <structfield>atthasmissing</structfield> is true. If there is no value
+ the column is null.
+ </entry>
+ </row>
+
</tbody>
</tgroup>
</table>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c7..edf459ae6b1 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1184,24 +1184,25 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</para>
<para>
- When a column is added with <literal>ADD COLUMN</literal>, all existing
- rows in the table are initialized with the column's default value
- (NULL if no <literal>DEFAULT</literal> clause is specified).
- If there is no <literal>DEFAULT</literal> clause, this is merely a metadata
- change and does not require any immediate update of the table's data;
- the added NULL values are supplied on readout, instead.
+ When a column is added with <literal>ADD COLUMN</literal> and a
+ non-volatile <literal>DEFAULT</literal> is specified, the default is
+ evaluated at the time of the statement and the result stored in the
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ neither case is a rewrite of the table required.
</para>
<para>
- Adding a column with a <literal>DEFAULT</literal> clause or changing the type of
- an existing column will require the entire table and its indexes to be
- rewritten. As an exception when changing the type of an existing column,
- if the <literal>USING</literal> clause does not change the column
- contents and the old type is either binary coercible to the new type or
- an unconstrained domain over the new type, a table rewrite is not needed;
- but any indexes on the affected columns must still be rebuilt. Adding or
- removing a system <literal>oid</literal> column also requires rewriting the entire
- table. Table and/or index rebuilds may take a significant amount of time
+ Adding a column with a volatile <literal>DEFAULT</literal> or
+ changing the type of an existing column will require the entire table and
+ its indexes to be rewritten. As an exception, when changing the type of an
+ existing column, if the <literal>USING</literal> clause does not change
+ the column contents and the old type is either binary coercible to the new
+ type or an unconstrained domain over the new type, a table rewrite is not
+ needed; but any indexes on the affected columns must still be rebuilt.
+ Adding or removing a system <literal>oid</literal> column also requires
+ rewriting the entire table.
+ Table and/or index rebuilds may take a significant amount of time
for a large table; and will temporarily require as much as double the disk
space.
</para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c45a48812bf..6d19537489c 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -58,6 +58,7 @@
#include "postgres.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -76,6 +77,73 @@
* ----------------------------------------------------------------
*/
+/*
+ * Return the missing value of an attribute, or NULL if there isn't one.
+ */
+static Datum
+getmissingattr(TupleDesc tupleDesc,
+ int attnum, bool *isnull)
+{
+ Form_pg_attribute att;
+ AttrMissing *attrmiss;
+
+ Assert(attnum <= tupleDesc->natts);
+ Assert(attnum > 0);
+
+ att = TupleDescAttr(tupleDesc, attnum - 1);
+
+ if (att->atthasmissing)
+ {
+ Assert(tupleDesc->constr);
+ Assert(tupleDesc->constr->missing);
+
+ attrmiss = tupleDesc->constr->missing + (attnum - 1);
+
+ if (attrmiss->ammissingPresent)
+ {
+ *isnull = false;
+ return attrmiss->ammissing;
+ }
+ }
+
+ *isnull = true;
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fill in missing values for a TupleTableSlot
+ */
+static void
+slot_getmissingattrs(TupleTableSlot *slot, int startAttNum, int lastAttNum)
+{
+ AttrMissing *attrmiss = NULL;
+ int missattnum;
+
+ if (slot->tts_tupleDescriptor->constr)
+ attrmiss = slot->tts_tupleDescriptor->constr->missing;
+
+
+ if (!attrmiss)
+ {
+ /* no missing values array at all, so just fill everything in as NULL */
+ memset(slot->tts_values + startAttNum, 0,
+ (lastAttNum - startAttNum) * sizeof(Datum));
+ memset(slot->tts_isnull + startAttNum, 1,
+ (lastAttNum - startAttNum) * sizeof(bool));
+ }
+ else
+ {
+ /* if there is a missing values array we must process them one by one */
+ for (missattnum = lastAttNum - 1;
+ missattnum >= startAttNum;
+ missattnum--)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] =
+ !attrmiss[missattnum].ammissingPresent;
+ }
+ }
+}
/*
* heap_compute_data_size
@@ -132,6 +200,134 @@ heap_compute_data_size(TupleDesc tupleDesc,
return data_length;
}
+/*
+ * Per-attribute helper for heap_fill_tuple and other routines building tuples.
+ *
+ * Fill in either a data value or a bit in the null bitmask
+ */
+static inline void
+fill_val(Form_pg_attribute att,
+ bits8 **bit,
+ int *bitmask,
+ char **dataP,
+ uint16 *infomask,
+ Datum datum,
+ bool isnull)
+{
+ Size data_length;
+ char *data = *dataP;
+
+ /*
+ * If we're building a null bitmap, set the appropriate bit for the
+ * current column value here.
+ */
+ if (bit != NULL)
+ {
+ if (*bitmask != HIGHBIT)
+ *bitmask <<= 1;
+ else
+ {
+ *bit += 1;
+ **bit = 0x0;
+ *bitmask = 1;
+ }
+
+ if (isnull)
+ {
+ *infomask |= HEAP_HASNULL;
+ return;
+ }
+
+ **bit |= *bitmask;
+ }
+
+ /*
+ * XXX we use the att_align macros on the pointer value itself, not on an
+ * offset. This is a bit of a hack.
+ */
+ if (att->attbyval)
+ {
+ /* pass-by-value */
+ data = (char *) att_align_nominal(data, att->attalign);
+ store_att_byval(data, datum, att->attlen);
+ data_length = att->attlen;
+ }
+ else if (att->attlen == -1)
+ {
+ /* varlena */
+ Pointer val = DatumGetPointer(datum);
+
+ *infomask |= HEAP_HASVARWIDTH;
+ if (VARATT_IS_EXTERNAL(val))
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED(val))
+ {
+ /*
+ * we want to flatten the expanded value so that the
+ * constructed tuple doesn't depend on it
+ */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(datum);
+
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = EOH_get_flat_size(eoh);
+ EOH_flatten_into(eoh, data, data_length);
+ }
+ else
+ {
+ *infomask |= HEAP_HASEXTERNAL;
+ /* no alignment, since it's short by definition */
+ data_length = VARSIZE_EXTERNAL(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (VARATT_IS_SHORT(val))
+ {
+ /* no alignment for short varlenas */
+ data_length = VARSIZE_SHORT(val);
+ memcpy(data, val, data_length);
+ }
+ else if (VARLENA_ATT_IS_PACKABLE(att) &&
+ VARATT_CAN_MAKE_SHORT(val))
+ {
+ /* convert to short varlena -- no alignment */
+ data_length = VARATT_CONVERTED_SHORT_SIZE(val);
+ SET_VARSIZE_SHORT(data, data_length);
+ memcpy(data + 1, VARDATA(val), data_length - 1);
+ }
+ else
+ {
+ /* full 4-byte header varlena */
+ data = (char *) att_align_nominal(data,
+ att->attalign);
+ data_length = VARSIZE(val);
+ memcpy(data, val, data_length);
+ }
+ }
+ else if (att->attlen == -2)
+ {
+ /* cstring ... never needs alignment */
+ *infomask |= HEAP_HASVARWIDTH;
+ Assert(att->attalign == 'c');
+ data_length = strlen(DatumGetCString(datum)) + 1;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+ else
+ {
+ /* fixed-length pass-by-reference */
+ data = (char *) att_align_nominal(data, att->attalign);
+ Assert(att->attlen > 0);
+ data_length = att->attlen;
+ memcpy(data, DatumGetPointer(datum), data_length);
+ }
+
+ data += data_length;
+ *dataP = data;
+
+
+
+}
+
/*
* heap_fill_tuple
* Load data portion of a tuple from values/isnull arrays
@@ -172,111 +368,15 @@ heap_fill_tuple(TupleDesc tupleDesc,
for (i = 0; i < numberOfAttributes; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
- Size data_length;
-
- if (bit != NULL)
- {
- if (bitmask != HIGHBIT)
- bitmask <<= 1;
- else
- {
- bitP += 1;
- *bitP = 0x0;
- bitmask = 1;
- }
-
- if (isnull[i])
- {
- *infomask |= HEAP_HASNULL;
- continue;
- }
-
- *bitP |= bitmask;
- }
-
- /*
- * XXX we use the att_align macros on the pointer value itself, not on
- * an offset. This is a bit of a hack.
- */
-
- if (att->attbyval)
- {
- /* pass-by-value */
- data = (char *) att_align_nominal(data, att->attalign);
- store_att_byval(data, values[i], att->attlen);
- data_length = att->attlen;
- }
- else if (att->attlen == -1)
- {
- /* varlena */
- Pointer val = DatumGetPointer(values[i]);
-
- *infomask |= HEAP_HASVARWIDTH;
- if (VARATT_IS_EXTERNAL(val))
- {
- if (VARATT_IS_EXTERNAL_EXPANDED(val))
- {
- /*
- * we want to flatten the expanded value so that the
- * constructed tuple doesn't depend on it
- */
- ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
-
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = EOH_get_flat_size(eoh);
- EOH_flatten_into(eoh, data, data_length);
- }
- else
- {
- *infomask |= HEAP_HASEXTERNAL;
- /* no alignment, since it's short by definition */
- data_length = VARSIZE_EXTERNAL(val);
- memcpy(data, val, data_length);
- }
- }
- else if (VARATT_IS_SHORT(val))
- {
- /* no alignment for short varlenas */
- data_length = VARSIZE_SHORT(val);
- memcpy(data, val, data_length);
- }
- else if (VARLENA_ATT_IS_PACKABLE(att) &&
- VARATT_CAN_MAKE_SHORT(val))
- {
- /* convert to short varlena -- no alignment */
- data_length = VARATT_CONVERTED_SHORT_SIZE(val);
- SET_VARSIZE_SHORT(data, data_length);
- memcpy(data + 1, VARDATA(val), data_length - 1);
- }
- else
- {
- /* full 4-byte header varlena */
- data = (char *) att_align_nominal(data,
- att->attalign);
- data_length = VARSIZE(val);
- memcpy(data, val, data_length);
- }
- }
- else if (att->attlen == -2)
- {
- /* cstring ... never needs alignment */
- *infomask |= HEAP_HASVARWIDTH;
- Assert(att->attalign == 'c');
- data_length = strlen(DatumGetCString(values[i])) + 1;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
- else
- {
- /* fixed-length pass-by-reference */
- data = (char *) att_align_nominal(data, att->attalign);
- Assert(att->attlen > 0);
- data_length = att->attlen;
- memcpy(data, DatumGetPointer(values[i]), data_length);
- }
-
- data += data_length;
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
+
+ fill_val(attr,
+ bitP ? &bitP : NULL,
+ &bitmask,
+ &data,
+ infomask,
+ values ? values[i] : PointerGetDatum(NULL),
+ isnull ? isnull[i] : true);
}
Assert((data - start) == data_size);
@@ -293,10 +393,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
* ----------------
*/
bool
-heap_attisnull(HeapTuple tup, int attnum)
+heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc)
{
+ /*
+ * We allow a NULL tupledesc for relations not expected to have missing
+ * values, such as catalog relations and indexes.
+ */
+ Assert(!tupleDesc || attnum <= tupleDesc->natts);
if (attnum > (int) HeapTupleHeaderGetNatts(tup->t_data))
- return true;
+ {
+ if (tupleDesc && TupleDescAttr(tupleDesc, attnum - 1)->atthasmissing)
+ return false;
+ else
+ return true;
+ }
if (attnum > 0)
{
@@ -649,6 +759,274 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
}
+/*
+ * Expand a tuple which has less attributes than required. For each attribute
+ * not present in the sourceTuple, if there is a missing value that will be
+ * used. Otherwise the attribute will be set to NULL.
+ *
+ * The source tuple must have less attributes than the required number.
+ *
+ * Only one of targetHeapTuple and targetMinimalTuple may be supplied. The
+ * other argument must be NULL.
+ */
+static void
+expand_tuple(HeapTuple *targetHeapTuple,
+ MinimalTuple *targetMinimalTuple,
+ HeapTuple sourceTuple,
+ TupleDesc tupleDesc)
+{
+ AttrMissing *attrmiss = NULL;
+ int attnum;
+ int firstmissingnum = 0;
+ bool hasNulls = HeapTupleHasNulls(sourceTuple);
+ HeapTupleHeader targetTHeader;
+ HeapTupleHeader sourceTHeader = sourceTuple->t_data;
+ int sourceNatts = HeapTupleHeaderGetNatts(sourceTHeader);
+ int natts = tupleDesc->natts;
+ int sourceNullLen;
+ int targetNullLen;
+ Size sourceDataLen = sourceTuple->t_len - sourceTHeader->t_hoff;
+ Size targetDataLen;
+ Size len;
+ int hoff;
+ bits8 *nullBits = NULL;
+ int bitMask = 0;
+ char *targetData;
+ uint16 *infoMask;
+
+ Assert((targetHeapTuple && !targetMinimalTuple)
+ || (!targetHeapTuple && targetMinimalTuple));
+
+ Assert(sourceNatts < natts);
+
+ sourceNullLen = (hasNulls ? BITMAPLEN(sourceNatts) : 0);
+
+ targetDataLen = sourceDataLen;
+
+ if (tupleDesc->constr &&
+ tupleDesc->constr->missing)
+ {
+ /*
+ * If there are missing values we want to put them into the tuple.
+ * Before that we have to compute the extra length for the values
+ * array and the variable length data.
+ */
+ attrmiss = tupleDesc->constr->missing;
+
+ /*
+ * Find the first item in attrmiss for which we don't have a value in
+ * the source. We can ignore all the missing entries before that.
+ */
+ for (firstmissingnum = sourceNatts;
+ firstmissingnum < natts;
+ firstmissingnum++)
+ {
+ if (attrmiss[firstmissingnum].ammissingPresent)
+ break;
+ }
+
+ /*
+ * If there are no more missing values everything else must be NULL
+ */
+ if (firstmissingnum >= natts)
+ {
+ hasNulls = true;
+ }
+ else
+ {
+
+ /*
+ * Now walk the missing attributes. If there is a missing value
+ * make space for it. Otherwise, it's going to be NULL.
+ */
+ for (attnum = firstmissingnum;
+ attnum < natts;
+ attnum++)
+ {
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ Form_pg_attribute att = TupleDescAttr(tupleDesc, attnum);
+
+ targetDataLen = att_align_datum(targetDataLen,
+ att->attalign,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+
+ targetDataLen = att_addlength_pointer(targetDataLen,
+ att->attlen,
+ attrmiss[attnum].ammissing);
+ }
+ else
+ {
+ /* no missing value, so it must be null */
+ hasNulls = true;
+ }
+ }
+ }
+ } /* end if have missing values */
+ else
+ {
+ /*
+ * If there are no missing values at all then NULLS must be allowed,
+ * since some of the attributes are known to be absent.
+ */
+ hasNulls = true;
+ }
+
+ len = 0;
+
+ if (hasNulls)
+ {
+ targetNullLen = BITMAPLEN(natts);
+ len += targetNullLen;
+ }
+ else
+ targetNullLen = 0;
+
+ if (tupleDesc->tdhasoid)
+ len += sizeof(Oid);
+
+ /*
+ * Allocate and zero the space needed. Note that the tuple body and
+ * HeapTupleData management structure are allocated in one chunk.
+ */
+ if (targetHeapTuple)
+ {
+ len += offsetof(HeapTupleHeaderData, t_bits);
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetHeapTuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
+ (*targetHeapTuple)->t_data
+ = targetTHeader
+ = (HeapTupleHeader) ((char *) *targetHeapTuple + HEAPTUPLESIZE);
+ (*targetHeapTuple)->t_len = len;
+ (*targetHeapTuple)->t_tableOid = sourceTuple->t_tableOid;
+ ItemPointerSetInvalid(&((*targetHeapTuple)->t_self));
+
+ targetTHeader->t_infomask = sourceTHeader->t_infomask;
+ targetTHeader->t_hoff = hoff;
+ HeapTupleHeaderSetNatts(targetTHeader, natts);
+ HeapTupleHeaderSetDatumLength(targetTHeader, len);
+ HeapTupleHeaderSetTypeId(targetTHeader, tupleDesc->tdtypeid);
+ HeapTupleHeaderSetTypMod(targetTHeader, tupleDesc->tdtypmod);
+ /* We also make sure that t_ctid is invalid unless explicitly set */
+ ItemPointerSetInvalid(&(targetTHeader->t_ctid));
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) (*targetHeapTuple)->t_data
+ + offsetof(HeapTupleHeaderData, t_bits));
+ targetData = (char *) (*targetHeapTuple)->t_data + hoff;
+ infoMask = &(targetTHeader->t_infomask);
+ }
+ else
+ {
+ len += SizeofMinimalTupleHeader;
+ hoff = len = MAXALIGN(len); /* align user data safely */
+ len += targetDataLen;
+
+ *targetMinimalTuple = (MinimalTuple) palloc0(len);
+ (*targetMinimalTuple)->t_len = len;
+ (*targetMinimalTuple)->t_hoff = hoff + MINIMAL_TUPLE_OFFSET;
+ (*targetMinimalTuple)->t_infomask = sourceTHeader->t_infomask;
+ /* Same macro works for MinimalTuples */
+ HeapTupleHeaderSetNatts(*targetMinimalTuple, natts);
+ if (targetNullLen > 0)
+ nullBits = (bits8 *) ((char *) *targetMinimalTuple
+ + offsetof(MinimalTupleData, t_bits));
+ targetData = (char *) *targetMinimalTuple + hoff;
+ infoMask = &((*targetMinimalTuple)->t_infomask);
+ }
+
+ if (targetNullLen > 0)
+ {
+ if (sourceNullLen > 0)
+ {
+ /* if bitmap pre-existed copy in - all is set */
+ memcpy(nullBits,
+ ((char *) sourceTHeader)
+ + offsetof(HeapTupleHeaderData, t_bits),
+ sourceNullLen);
+ nullBits += sourceNullLen - 1;
+ }
+ else
+ {
+ sourceNullLen = BITMAPLEN(sourceNatts);
+ /* Set NOT NULL for all existing attributes */
+ memset(nullBits, 0xff, sourceNullLen);
+
+ nullBits += sourceNullLen - 1;
+
+ if (sourceNatts & 0x07)
+ {
+ /* build the mask (inverted!) */
+ bitMask = 0xff << (sourceNatts & 0x07);
+ /* Voila */
+ *nullBits = ~bitMask;
+ }
+ }
+
+ bitMask = (1 << ((sourceNatts - 1) & 0x07));
+ } /* End if have null bitmap */
+
+ memcpy(targetData,
+ ((char *) sourceTuple->t_data) + sourceTHeader->t_hoff,
+ sourceDataLen);
+
+ targetData += sourceDataLen;
+
+ /* Now fill in the missing values */
+ for (attnum = sourceNatts; attnum < natts; attnum++)
+ {
+
+ Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+
+ if (attrmiss[attnum].ammissingPresent)
+ {
+ fill_val(attr,
+ nullBits ? &nullBits : NULL,
+ &bitMask,
+ &targetData,
+ infoMask,
+ attrmiss[attnum].ammissing,
+ false);
+ }
+ else
+ {
+ fill_val(attr,
+ &nullBits,
+ &bitMask,
+ &targetData,
+ infoMask,
+ (Datum) 0,
+ true);
+ }
+ } /* end loop over missing attributes */
+}
+
+/*
+ * Fill in the missing values for a minimal HeapTuple
+ */
+MinimalTuple
+minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ MinimalTuple minimalTuple;
+
+ expand_tuple(NULL, &minimalTuple, sourceTuple, tupleDesc);
+ return minimalTuple;
+}
+
+/*
+ * Fill in the missing values for an ordinary HeapTuple
+ */
+HeapTuple
+heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
+{
+ HeapTuple heapTuple;
+
+ expand_tuple(&heapTuple, NULL, sourceTuple, tupleDesc);
+ return heapTuple;
+}
+
/* ----------------
* heap_copy_tuple_as_datum
*
@@ -1012,13 +1390,10 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as nulls or missing values as appropriate.
*/
for (; attnum < tdesc_natts; attnum++)
- {
- values[attnum] = (Datum) 0;
- isnull[attnum] = true;
- }
+ values[attnum] = getmissingattr(tupleDesc, attnum + 1, &isnull[attnum]);
}
/*
@@ -1192,8 +1567,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
{
- *isnull = true;
- return (Datum) 0;
+ return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
}
/*
@@ -1263,13 +1637,11 @@ slot_getallattrs(TupleTableSlot *slot)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLS or missing values.
*/
- for (; attnum < tdesc_natts; attnum++)
- {
- slot->tts_values[attnum] = (Datum) 0;
- slot->tts_isnull[attnum] = true;
- }
+ if (attnum < tdesc_natts)
+ slot_getmissingattrs(slot, attnum, tdesc_natts);
+
slot->tts_nvalid = tdesc_natts;
}
@@ -1310,13 +1682,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
@@ -1340,7 +1710,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract system attribute from virtual tuple");
if (tuple == &(slot->tts_minhdr)) /* internal error */
elog(ERROR, "cannot extract system attribute from minimal tuple");
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
@@ -1363,7 +1733,7 @@ slot_attisnull(TupleTableSlot *slot, int attnum)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/* and let the tuple tell it */
- return heap_attisnull(tuple, attnum);
+ return heap_attisnull(tuple, attnum, tupleDesc);
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd7..54393607222 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,12 +21,14 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/hashutils.h"
#include "utils/resowner_private.h"
#include "utils/syscache.h"
@@ -176,6 +178,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
+ if (constr->missing)
+ {
+ cpy->missing = (AttrMissing *) palloc(tupdesc->natts * sizeof(AttrMissing));
+ memcpy(cpy->missing, constr->missing, tupdesc->natts * sizeof(AttrMissing));
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (constr->missing[i].ammissingPresent)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+
+ cpy->missing[i].ammissing = datumCopy(constr->missing[i].ammissing,
+ attr->attbyval,
+ attr->attlen);
+ }
+ }
+ }
+
if ((cpy->num_check = constr->num_check) > 0)
{
cpy->check = (ConstrCheck *) palloc(cpy->num_check * sizeof(ConstrCheck));
@@ -309,6 +328,18 @@ FreeTupleDesc(TupleDesc tupdesc)
}
pfree(attrdef);
}
+ if (tupdesc->constr->missing)
+ {
+ AttrMissing *attrmiss = tupdesc->constr->missing;
+
+ for (i = tupdesc->natts - 1; i >= 0; i--)
+ {
+ if (attrmiss[i].ammissingPresent
+ && !TupleDescAttr(tupdesc, i)->attbyval)
+ pfree(DatumGetPointer(attrmiss[i].ammissing));
+ }
+ pfree(attrmiss);
+ }
if (tupdesc->constr->num_check > 0)
{
ConstrCheck *check = tupdesc->constr->check;
@@ -469,6 +500,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
+
+ if (missval1->ammissingPresent != missval2->ammissingPresent)
+ return false;
+ if (missval1->ammissingPresent)
+ {
+ Form_pg_attribute missatt1 = TupleDescAttr(tupdesc1, i);
+
+ if (!datumIsEqual(missval1->ammissing, missval2->ammissing,
+ missatt1->attbyval, missatt1->attlen))
+ return false;
+ }
+ }
+ }
+ else if (constr2->missing)
+ return false;
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
@@ -584,6 +638,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
@@ -797,6 +852,7 @@ BuildDescForRelation(List *schema)
constr->has_not_null = true;
constr->defval = NULL;
+ constr->missing = NULL;
constr->num_defval = 0;
constr->check = NULL;
constr->num_check = 0;
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 06485397969..83000575ce7 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -4588,7 +4588,7 @@ pg_attribute_aclcheck_all(Oid table_oid, Oid roleid, AclMode mode,
* grants no privileges, so that we can fall out quickly in the very
* common case where attacl is null.
*/
- if (heap_attisnull(attTuple, Anum_pg_attribute_attacl))
+ if (heap_attisnull(attTuple, Anum_pg_attribute_attacl, NULL))
attmask = 0;
else
attmask = pg_attribute_aclmask(table_oid, curr_att, roleid,
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b69bb1e2a41..2b34bb8313d 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -60,9 +60,12 @@
#include "catalog/storage_xlog.h"
#include "commands/tablecmds.h"
#include "commands/typecmds.h"
+#include "executor/executor.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
+#include "optimizer/clauses.h"
#include "optimizer/var.h"
+#include "optimizer/planner.h"
#include "parser/parse_coerce.h"
#include "parser/parse_collate.h"
#include "parser/parse_expr.h"
@@ -72,6 +75,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -144,37 +148,37 @@ static List *insert_ordered_unique_oid(List *list, Oid datum);
static FormData_pg_attribute a1 = {
0, {"ctid"}, TIDOID, 0, sizeof(ItemPointerData),
SelfItemPointerAttributeNumber, 0, -1, -1,
- false, 'p', 's', true, false, '\0', false, true, 0
+ false, 'p', 's', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a2 = {
0, {"oid"}, OIDOID, 0, sizeof(Oid),
ObjectIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a3 = {
0, {"xmin"}, XIDOID, 0, sizeof(TransactionId),
MinTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a4 = {
0, {"cmin"}, CIDOID, 0, sizeof(CommandId),
MinCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a5 = {
0, {"xmax"}, XIDOID, 0, sizeof(TransactionId),
MaxTransactionIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static FormData_pg_attribute a6 = {
0, {"cmax"}, CIDOID, 0, sizeof(CommandId),
MaxCommandIdAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
/*
@@ -186,7 +190,7 @@ static FormData_pg_attribute a6 = {
static FormData_pg_attribute a7 = {
0, {"tableoid"}, OIDOID, 0, sizeof(Oid),
TableOidAttributeNumber, 0, -1, -1,
- true, 'p', 'i', true, false, '\0', false, true, 0
+ true, 'p', 'i', true, false, false, '\0', false, true, 0
};
static const Form_pg_attribute SysAtt[] = {&a1, &a2, &a3, &a4, &a5, &a6, &a7};
@@ -624,6 +628,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attalign - 1] = CharGetDatum(new_attribute->attalign);
values[Anum_pg_attribute_attnotnull - 1] = BoolGetDatum(new_attribute->attnotnull);
values[Anum_pg_attribute_atthasdef - 1] = BoolGetDatum(new_attribute->atthasdef);
+ values[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(new_attribute->atthasmissing);
values[Anum_pg_attribute_attidentity - 1] = CharGetDatum(new_attribute->attidentity);
values[Anum_pg_attribute_attisdropped - 1] = BoolGetDatum(new_attribute->attisdropped);
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
@@ -634,6 +639,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
nulls[Anum_pg_attribute_attacl - 1] = true;
nulls[Anum_pg_attribute_attoptions - 1] = true;
nulls[Anum_pg_attribute_attfdwoptions - 1] = true;
+ nulls[Anum_pg_attribute_attmissingval - 1] = true;
tup = heap_form_tuple(RelationGetDescr(pg_attribute_rel), values, nulls);
@@ -1925,14 +1931,92 @@ heap_drop_with_catalog(Oid relid)
}
+/*
+ * RelationClearMissing
+ *
+ * set atthasmissing and attmissingval to false/null for all attributes
+ * where they are currently set. This can be safely and usefully done if
+ * the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
+ * are no rows left with less than a full complement of attributes.
+ *
+ * The caller must have an AccessExclusive lock on the relation.
+ */
+
+void
+RelationClearMissing(Relation rel)
+{
+ Relation attr_rel;
+ Oid relid = RelationGetRelid(rel);
+ int natts = RelationGetNumberOfAttributes(rel);
+ int attnum;
+ Datum repl_val[Natts_pg_attribute];
+ bool repl_null[Natts_pg_attribute];
+ bool repl_repl[Natts_pg_attribute];
+ Form_pg_attribute attrtuple;
+ HeapTuple tuple,
+ newtuple;
+
+ memset(repl_val, 0, sizeof(repl_val));
+ memset(repl_null, false, sizeof(repl_null));
+ memset(repl_repl, false, sizeof(repl_repl));
+
+ repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false);
+ repl_null[Anum_pg_attribute_attmissingval - 1] = true;
+
+ repl_repl[Anum_pg_attribute_atthasmissing - 1] = true;
+ repl_repl[Anum_pg_attribute_attmissingval - 1] = true;
+
+
+ /* Get a lock on pg_attribute */
+ attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ /* process each non-system attribute */
+ for (attnum = 1; attnum <= natts; attnum++)
+ {
+ tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+ if (!HeapTupleIsValid(tuple)) /* shouldn't happen */
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /* ignore any where atthasmissing is not true */
+ if (attrtuple->atthasmissing)
+ {
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attr_rel),
+ repl_val, repl_null, repl_repl);
+
+ CatalogTupleUpdate(attr_rel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+
+ ReleaseSysCache(tuple);
+ }
+
+ /*
+ * Our update of the pg_attribute rows will force a relcache rebuild, so
+ * there's nothing else to do here.
+ */
+ heap_close(attr_rel, RowExclusiveLock);
+}
+
/*
* Store a default expression for column attnum of relation rel.
*
* Returns the OID of the new pg_attrdef tuple.
+ *
+ * add_column_mode must be true if we are storing the default for a new
+ * attribute, and false if it's for an already existing attribute. The reason
+ * for this is that the missing value must never be updated after it is set,
+ * which can only be when a column is added to the table. Otherwise we would
+ * in effect be changing existing tuples.
*/
Oid
StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal)
+ Node *expr, bool is_internal, bool add_column_mode)
{
char *adbin;
char *adsrc;
@@ -1943,9 +2027,19 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
+ Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -2000,7 +2094,50 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
- attStruct->atthasdef = true;
+ MemSet(valuesAtt, 0, sizeof(valuesAtt));
+ MemSet(nullsAtt, false, sizeof(nullsAtt));
+ MemSet(replacesAtt, false, sizeof(replacesAtt));
+ valuesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+ replacesAtt[Anum_pg_attribute_atthasdef - 1] = true;
+
+ if (add_column_mode)
+ {
+ expr2 = expression_planner(expr2);
+ estate = CreateExecutorState();
+ exprState = ExecPrepareExpr(expr2, estate);
+ econtext = GetPerTupleExprContext(estate);
+
+ missingval = ExecEvalExpr(exprState, econtext,
+ &missingIsNull);
+
+ defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
+
+ if (missingIsNull)
+ {
+ /* if the default evaluates to NULL, just store a NULL array */
+ missingval = (Datum) 0;
+ }
+ else
+ {
+ /* otherwise make a one-element array of the value */
+ missingval = PointerGetDatum(
+ construct_array(&missingval,
+ 1,
+ defAttStruct->atttypid,
+ defAttStruct->attlen,
+ defAttStruct->attbyval,
+ defAttStruct->attalign));
+ }
+
+ valuesAtt[Anum_pg_attribute_atthasmissing - 1] = !missingIsNull;
+ replacesAtt[Anum_pg_attribute_atthasmissing - 1] = true;
+ valuesAtt[Anum_pg_attribute_attmissingval - 1] = missingval;
+ replacesAtt[Anum_pg_attribute_attmissingval - 1] = true;
+ nullsAtt[Anum_pg_attribute_attmissingval - 1] = missingIsNull;
+ }
+ atttup = heap_modify_tuple(atttup, RelationGetDescr(attrrel),
+ valuesAtt, nullsAtt, replacesAtt);
+
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
}
heap_close(attrrel, RowExclusiveLock);
@@ -2032,6 +2169,14 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
+ if (estate)
+ {
+ FreeExecutorState(estate);
+ }
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
return attrdefOid;
}
@@ -2185,7 +2330,7 @@ StoreConstraints(Relation rel, List *cooked_constraints, bool is_internal)
{
case CONSTR_DEFAULT:
con->conoid = StoreAttrDefault(rel, con->attnum, con->expr,
- is_internal);
+ is_internal, false);
break;
case CONSTR_CHECK:
con->conoid =
@@ -2301,7 +2446,14 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal);
+ /* If the default is volatile we cannot use a missing value */
+ if (colDef->missingMode && contain_volatile_functions((Node *) expr))
+ {
+ colDef->missingMode = false;
+ }
+
+ defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
+ colDef->missingMode);
cooked = (CookedConstraint *) palloc(sizeof(CookedConstraint));
cooked->contype = CONSTR_DEFAULT;
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bfac37f9d17..fc8ec14eeef 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -371,6 +371,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->attnotnull = false;
to->atthasdef = false;
+ to->atthasmissing = false;
to->attidentity = '\0';
to->attislocal = true;
to->attinhcount = 0;
@@ -1654,7 +1655,8 @@ index_drop(Oid indexId, bool concurrent)
if (!HeapTupleIsValid(tuple))
elog(ERROR, "cache lookup failed for index %u", indexId);
- hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs);
+ hasexprs = !heap_attisnull(tuple, Anum_pg_index_indexprs,
+ RelationGetDescr(indexRelation));
CatalogTupleDelete(indexRelation, &tuple->t_self);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 57f3917fdc4..639b6992d53 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -453,7 +453,7 @@ check_index_is_clusterable(Relation OldHeap, Oid indexOid, bool recheck, LOCKMOD
* seqscan pass over the table to copy the missing rows, but that seems
* expensive and tedious.
*/
- if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred))
+ if (!heap_attisnull(OldIndex->rd_indextuple, Anum_pg_index_indpred, NULL))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot cluster on partial index \"%s\"",
@@ -1669,6 +1669,16 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
}
relation_close(newrel, NoLock);
}
+
+ /* if it's not a catalog table, clear any missing attribute settings */
+ if (!is_system_catalog)
+ {
+ Relation newrel;
+
+ newrel = heap_open(OIDOldHeap, NoLock);
+ RelationClearMissing(newrel);
+ relation_close(newrel, NoLock);
+ }
}
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index 86fa8c0dd74..c46493dd88b 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -2252,7 +2252,7 @@ ExecuteCallStmt(CallStmt *stmt, ParamListInfo params, bool atomic, DestReceiver
tp = SearchSysCache1(PROCOID, ObjectIdGetDatum(fexpr->funcid));
if (!HeapTupleIsValid(tp))
elog(ERROR, "cache lookup failed for function %u", fexpr->funcid);
- if (!heap_attisnull(tp, Anum_pg_proc_proconfig))
+ if (!heap_attisnull(tp, Anum_pg_proc_proconfig, NULL))
callcontext->atomic = true;
ReleaseSysCache(tp);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 0a2ab500238..01859707940 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -215,8 +215,8 @@ CheckIndexCompatible(Oid oldId,
* We don't assess expressions or predicates; assume incompatibility.
* Also, if the index is invalid for any reason, treat it as incompatible.
*/
- if (!(heap_attisnull(tuple, Anum_pg_index_indpred) &&
- heap_attisnull(tuple, Anum_pg_index_indexprs) &&
+ if (!(heap_attisnull(tuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(tuple, Anum_pg_index_indexprs, NULL) &&
IndexIsValid(indexForm)))
{
ReleaseSysCache(tuple);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e74fb1f4691..c0790a724f8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -714,6 +714,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = colDef->raw_default;
+ rawEnt->missingMode = false;
rawDefaults = lappend(rawDefaults, rawEnt);
attr->atthasdef = true;
}
@@ -4682,7 +4683,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
{
int attn = lfirst_int(l);
- if (heap_attisnull(tuple, attn + 1))
+ if (heap_attisnull(tuple, attn + 1, newTupDesc))
{
Form_pg_attribute attr = TupleDescAttr(newTupDesc, attn);
@@ -4785,7 +4786,7 @@ ATGetQueueEntry(List **wqueue, Relation rel)
tab = (AlteredTableInfo *) palloc0(sizeof(AlteredTableInfo));
tab->relid = relid;
tab->relkind = rel->rd_rel->relkind;
- tab->oldDesc = CreateTupleDescCopy(RelationGetDescr(rel));
+ tab->oldDesc = CreateTupleDescCopyConstr(RelationGetDescr(rel));
tab->newrelpersistence = RELPERSISTENCE_PERMANENT;
tab->chgPersistence = false;
@@ -5404,6 +5405,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attalign = tform->typalign;
attribute.attnotnull = colDef->is_not_null;
attribute.atthasdef = false;
+ attribute.atthasmissing = false;
attribute.attidentity = colDef->identity;
attribute.attisdropped = false;
attribute.attislocal = colDef->is_local;
@@ -5447,6 +5449,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+ rawEnt->missingMode = true;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -5457,6 +5460,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* Make the additional catalog changes visible */
CommandCounterIncrement();
+
+ /*
+ * Did the request for a missing value work? If not we'll have to do
+ * a rewrite
+ */
+ if (!rawEnt->missingMode)
+ {
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+ }
}
/*
@@ -5502,6 +5514,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
nve->typeId = typeOid;
defval = (Expr *) nve;
+
+ /* must do a rewrite for identity columns */
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
else
defval = (Expr *) build_column_default(rel, attribute.attnum);
@@ -5537,16 +5552,21 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
newval->expr = expression_planner(defval);
tab->newvals = lappend(tab->newvals, newval);
- tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
}
- /*
- * If the new column is NOT NULL, tell Phase 3 it needs to test that.
- * (Note we don't do this for an OID column. OID will be marked not
- * null, but since it's filled specially, there's no need to test
- * anything.)
- */
- tab->new_notnull |= colDef->is_not_null;
+ if (DomainHasConstraints(typeOid))
+ tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
+
+ if (!TupleDescAttr(rel->rd_att, attribute.attnum - 1)->atthasmissing)
+ {
+ /*
+ * If the new column is NOT NULL, and there is no missing value,
+ * tell Phase 3 it needs to test that. (Note we don't do this for
+ * an OID column. OID will be marked not null, but since it's
+ * filled specially, there's no need to test anything.)
+ */
+ tab->new_notnull |= colDef->is_not_null;
+ }
}
/*
@@ -6022,6 +6042,7 @@ ATExecColumnDefault(Relation rel, const char *colName,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attnum;
rawEnt->raw_default = newDefault;
+ rawEnt->missingMode = false;
/*
* This function is intended for CREATE TABLE, so it processes a
@@ -8109,8 +8130,8 @@ transformFkeyCheckAttrs(Relation pkrel,
if (indexStruct->indnatts == numattrs &&
indexStruct->indisunique &&
IndexIsValid(indexStruct) &&
- heap_attisnull(indexTuple, Anum_pg_index_indpred) &&
- heap_attisnull(indexTuple, Anum_pg_index_indexprs))
+ heap_attisnull(indexTuple, Anum_pg_index_indpred, NULL) &&
+ heap_attisnull(indexTuple, Anum_pg_index_indexprs, NULL))
{
Datum indclassDatum;
bool isnull;
@@ -9516,7 +9537,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, true,
true);
- StoreAttrDefault(rel, attnum, defaultexpr, true);
+ StoreAttrDefault(rel, attnum, defaultexpr, true, false);
}
ObjectAddressSubSet(address, RelationRelationId,
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 25221965e9e..2fdcb7f3fd3 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -2397,7 +2397,7 @@ AlterDomainNotNull(List *names, bool notNull)
int attnum = rtc->atts[i];
Form_pg_attribute attr = TupleDescAttr(tupdesc, attnum - 1);
- if (heap_attisnull(tuple, attnum))
+ if (heap_attisnull(tuple, attnum, tupdesc))
{
/*
* In principle the auxiliary information for this
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index f7bcf6370b5..e530b262dae 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2505,7 +2505,7 @@ ExecEvalRowNullInt(ExprState *state, ExprEvalStep *op,
/* ignore dropped columns */
if (TupleDescAttr(tupDesc, att - 1)->attisdropped)
continue;
- if (heap_attisnull(&tmptup, att))
+ if (heap_attisnull(&tmptup, att, tupDesc))
{
/* null field disproves IS NOT NULL */
if (!checkisnull)
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 890067757c0..516dde45eb6 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -2978,8 +2978,17 @@ EvalPlanQualFetchRowMarks(EPQState *epqstate)
false, NULL))
elog(ERROR, "failed to fetch tuple for EvalPlanQual recheck");
- /* successful, copy tuple */
- copyTuple = heap_copytuple(&tuple);
+ if (HeapTupleHeaderGetNatts(tuple.t_data) <
+ RelationGetDescr(erm->relation)->natts)
+ {
+ copyTuple = heap_expand_tuple(&tuple,
+ RelationGetDescr(erm->relation));
+ }
+ else
+ {
+ /* successful, copy tuple */
+ copyTuple = heap_copytuple(&tuple);
+ }
ReleaseBuffer(buffer);
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index c46d65cf938..025d3e379c3 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
/*
* Otherwise we need to build a tuple from the Datum array.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext = MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
/*
* Otherwise materialize the slot...
diff --git a/src/backend/executor/execUtils.c b/src/backend/executor/execUtils.c
index 14b07b5d449..b963cae730c 100644
--- a/src/backend/executor/execUtils.c
+++ b/src/backend/executor/execUtils.c
@@ -511,6 +511,8 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
return false; /* out of order */
if (att_tup->attisdropped)
return false; /* table contains dropped columns */
+ if (att_tup->atthasmissing)
+ return false; /* table contains cols with missing values */
/*
* Note: usually the Var's type should match the tupdesc exactly, but
diff --git a/src/backend/optimizer/util/clauses.c b/src/backend/optimizer/util/clauses.c
index 93e5658a354..ed6b680ed86 100644
--- a/src/backend/optimizer/util/clauses.c
+++ b/src/backend/optimizer/util/clauses.c
@@ -4491,7 +4491,7 @@ inline_function(Oid funcid, Oid result_type, Oid result_collid,
funcform->prosecdef ||
funcform->proretset ||
funcform->prorettype == RECORDOID ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL) ||
funcform->pronargs != list_length(args))
return NULL;
@@ -5031,7 +5031,7 @@ inline_set_returning_function(PlannerInfo *root, RangeTblEntry *rte)
funcform->prorettype == VOIDOID ||
funcform->prosecdef ||
!funcform->proretset ||
- !heap_attisnull(func_tuple, Anum_pg_proc_proconfig))
+ !heap_attisnull(func_tuple, Anum_pg_proc_proconfig, NULL))
{
ReleaseSysCache(func_tuple);
return NULL;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index bd3a0c4a0ab..5348c77b23d 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1493,12 +1493,12 @@ relation_excluded_by_constraints(PlannerInfo *root,
* in order. The executor can special-case such tlists to avoid a projection
* step at runtime, so we use such tlists preferentially for scan nodes.
*
- * Exception: if there are any dropped columns, we punt and return NIL.
- * Ideally we would like to handle the dropped-column case too. However this
- * creates problems for ExecTypeFromTL, which may be asked to build a tupdesc
- * for a tlist that includes vars of no-longer-existent types. In theory we
- * could dig out the required info from the pg_attribute entries of the
- * relation, but that data is not readily available to ExecTypeFromTL.
+ * Exception: if there are any dropped or missing columns, we punt and return
+ * NIL. Ideally we would like to handle these cases too. However
+ * this creates problems for ExecTypeFromTL, which may be asked to build a
+ * tupdesc for a tlist that includes vars of no-longer-existent types. In
+ * theory we could dig out the required info from the pg_attribute entries of
+ * the relation, but that data is not readily available to ExecTypeFromTL.
* For now, we don't apply the physical-tlist optimization when there are
* dropped cols.
*
@@ -1533,9 +1533,9 @@ build_physical_tlist(PlannerInfo *root, RelOptInfo *rel)
Form_pg_attribute att_tup = TupleDescAttr(relation->rd_att,
attrno - 1);
- if (att_tup->attisdropped)
+ if (att_tup->attisdropped || att_tup->atthasmissing)
{
- /* found a dropped col, so punt */
+ /* found a dropped or missing col, so punt */
tlist = NIL;
break;
}
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 66253fc3d3e..361bde42614 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1126,7 +1126,8 @@ build_column_default(Relation rel, int attrno)
/*
* Scan to see if relation has a default for this column.
*/
- if (rd_att->constr && rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef && rd_att->constr &&
+ rd_att->constr->num_defval > 0)
{
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
diff --git a/src/backend/statistics/extended_stats.c b/src/backend/statistics/extended_stats.c
index d34d5c3d12e..7eb981a7ed7 100644
--- a/src/backend/statistics/extended_stats.c
+++ b/src/backend/statistics/extended_stats.c
@@ -158,7 +158,7 @@ statext_is_kind_built(HeapTuple htup, char type)
elog(ERROR, "unexpected statistics type requested: %d", type);
}
- return !heap_attisnull(htup, attnum);
+ return !heap_attisnull(htup, attnum, NULL);
}
/*
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 4d7fee0ecb9..3bb708f8638 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -204,7 +204,7 @@ static void ri_GenerateQual(StringInfo buf,
Oid opoid,
const char *rightop, Oid rightoptype);
static void ri_GenerateQualCollation(StringInfo buf, Oid collation);
-static int ri_NullCheck(HeapTuple tup,
+static int ri_NullCheck(TupleDesc tupdesc, HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk);
static void ri_BuildQueryKey(RI_QueryKey *key,
const RI_ConstraintInfo *riinfo,
@@ -307,7 +307,7 @@ RI_FKey_check(TriggerData *trigdata)
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("MATCH PARTIAL not yet implemented")));
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
@@ -514,7 +514,7 @@ ri_Check_Pk_Match(Relation pk_rel, Relation fk_rel,
bool result;
/* Only called for non-null rows */
- Assert(ri_NullCheck(old_row, riinfo, true) == RI_KEYS_NONE_NULL);
+ Assert(ri_NullCheck(RelationGetDescr(fk_rel), old_row, riinfo, true) == RI_KEYS_NONE_NULL);
if (SPI_connect() != SPI_OK_CONNECT)
elog(ERROR, "SPI_connect failed");
@@ -724,7 +724,7 @@ ri_restrict(TriggerData *trigdata, bool is_no_action)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -911,7 +911,7 @@ RI_FKey_cascade_del(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1071,7 +1071,7 @@ RI_FKey_cascade_upd(PG_FUNCTION_ARGS)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1285,7 +1285,7 @@ ri_setnull(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1501,7 +1501,7 @@ ri_setdefault(TriggerData *trigdata)
*/
case FKCONSTR_MATCH_SIMPLE:
case FKCONSTR_MATCH_FULL:
- switch (ri_NullCheck(old_row, riinfo, true))
+ switch (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true))
{
case RI_KEYS_ALL_NULL:
case RI_KEYS_SOME_NULL:
@@ -1676,7 +1676,7 @@ RI_FKey_pk_upd_check_required(Trigger *trigger, Relation pk_rel,
* If any old key value is NULL, the row could not have been
* referenced by an FK row, so no check is needed.
*/
- if (ri_NullCheck(old_row, riinfo, true) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(pk_rel), old_row, riinfo, true) != RI_KEYS_NONE_NULL)
return false;
/* If all old and new key values are equal, no check is needed */
@@ -1732,7 +1732,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* If any new key value is NULL, the row must satisfy the
* constraint, so no check is needed.
*/
- if (ri_NullCheck(new_row, riinfo, false) != RI_KEYS_NONE_NULL)
+ if (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false) != RI_KEYS_NONE_NULL)
return false;
/*
@@ -1763,7 +1763,7 @@ RI_FKey_fk_upd_check_required(Trigger *trigger, Relation fk_rel,
* invalidated before the constraint is to be checked, but we
* should queue the event to apply the check later.
*/
- switch (ri_NullCheck(new_row, riinfo, false))
+ switch (ri_NullCheck(RelationGetDescr(fk_rel), new_row, riinfo, false))
{
case RI_KEYS_ALL_NULL:
return false;
@@ -2057,7 +2057,7 @@ RI_Initial_Check(Trigger *trigger, Relation fk_rel, Relation pk_rel)
* disallows partially-null FK rows.
*/
if (fake_riinfo.confmatchtype == FKCONSTR_MATCH_FULL &&
- ri_NullCheck(tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
+ ri_NullCheck(tupdesc, tuple, &fake_riinfo, false) != RI_KEYS_NONE_NULL)
ereport(ERROR,
(errcode(ERRCODE_FOREIGN_KEY_VIOLATION),
errmsg("insert or update on table \"%s\" violates foreign key constraint \"%s\"",
@@ -2860,7 +2860,8 @@ ri_ReportViolation(const RI_ConstraintInfo *riinfo,
* ----------
*/
static int
-ri_NullCheck(HeapTuple tup,
+ri_NullCheck(TupleDesc tupDesc,
+ HeapTuple tup,
const RI_ConstraintInfo *riinfo, bool rel_is_pk)
{
const int16 *attnums;
@@ -2875,7 +2876,7 @@ ri_NullCheck(HeapTuple tup,
for (i = 0; i < riinfo->nkeys; i++)
{
- if (heap_attisnull(tup, attnums[i]))
+ if (heap_attisnull(tup, attnums[i], tupDesc))
nonenull = false;
else
allnull = false;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index b0559ca5bcd..f8fc7f83f9b 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -1242,7 +1242,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
* versions of the expressions and predicate, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
{
Datum exprsDatum;
bool isnull;
@@ -1410,7 +1410,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno,
/*
* If it's a partial index, decompile and append the predicate
*/
- if (!heap_attisnull(ht_idx, Anum_pg_index_indpred))
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
{
Node *node;
Datum predDatum;
@@ -1644,7 +1644,7 @@ pg_get_partkeydef_worker(Oid relid, int prettyFlags,
* versions of the expressions, because we want to display
* non-const-folded expressions.)
*/
- if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs))
+ if (!heap_attisnull(tuple, Anum_pg_partitioned_table_partexprs, NULL))
{
Datum exprsDatum;
bool isnull;
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db26bdd..db6d8b0ceb3 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -36,6 +36,7 @@
#include "access/nbtree.h"
#include "access/reloptions.h"
#include "access/sysattr.h"
+#include "access/tupdesc_details.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -493,6 +495,7 @@ RelationBuildTupleDesc(Relation relation)
int need;
TupleConstr *constr;
AttrDefault *attrdef = NULL;
+ AttrMissing *attrmiss = NULL;
int ndef = 0;
/* copy some fields from pg_class row to rd_att */
@@ -538,15 +541,17 @@ RelationBuildTupleDesc(Relation relation)
while (HeapTupleIsValid(pg_attribute_tuple = systable_getnext(pg_attribute_scan)))
{
Form_pg_attribute attp;
+ int attnum;
attp = (Form_pg_attribute) GETSTRUCT(pg_attribute_tuple);
- if (attp->attnum <= 0 ||
- attp->attnum > relation->rd_rel->relnatts)
+ attnum = attp->attnum;
+ if (attnum <= 0 || attnum > relation->rd_rel->relnatts)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
- memcpy(TupleDescAttr(relation->rd_att, attp->attnum - 1),
+
+ memcpy(TupleDescAttr(relation->rd_att, attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -554,6 +559,7 @@ RelationBuildTupleDesc(Relation relation)
if (attp->attnotnull)
constr->has_not_null = true;
+ /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
{
if (attrdef == NULL)
@@ -561,10 +567,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
@@ -605,7 +664,8 @@ RelationBuildTupleDesc(Relation relation)
/*
* Set up constraint/default info
*/
- if (constr->has_not_null || ndef > 0 || relation->rd_rel->relchecks)
+ if (constr->has_not_null || ndef > 0 ||
+ attrmiss || relation->rd_rel->relchecks)
{
relation->rd_att->constr = constr;
@@ -620,7 +680,10 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
+ {
constr->num_defval = 0;
+ }
+ constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
{
@@ -4056,10 +4119,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}
/*
@@ -4398,7 +4457,7 @@ RelationGetIndexList(Relation relation)
*/
if (!IndexIsValid(index) || !index->indisunique ||
!index->indimmediate ||
- !heap_attisnull(htup, Anum_pg_index_indpred))
+ !heap_attisnull(htup, Anum_pg_index_indpred, NULL))
continue;
/* Check to see if is a usable btree index on OID */
@@ -4693,7 +4752,7 @@ RelationGetIndexExpressions(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indexprs, NULL))
return NIL;
/*
@@ -4755,7 +4814,7 @@ RelationGetIndexPredicate(Relation relation)
/* Quick exit if there is nothing to do. */
if (relation->rd_indextuple == NULL ||
- heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred))
+ heap_attisnull(relation->rd_indextuple, Anum_pg_index_indpred, NULL))
return NIL;
/*
diff --git a/src/backend/utils/fmgr/fmgr.c b/src/backend/utils/fmgr/fmgr.c
index ae511e9a23b..aa188bdeffa 100644
--- a/src/backend/utils/fmgr/fmgr.c
+++ b/src/backend/utils/fmgr/fmgr.c
@@ -200,7 +200,7 @@ fmgr_info_cxt_security(Oid functionId, FmgrInfo *finfo, MemoryContext mcxt,
*/
if (!ignore_security &&
(procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId)))
{
finfo->fn_addr = fmgr_security_definer;
@@ -294,7 +294,7 @@ fmgr_symbol(Oid functionId, char **mod, char **fn)
/*
*/
if (procedureStruct->prosecdef ||
- !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig) ||
+ !heap_attisnull(procedureTuple, Anum_pg_proc_proconfig, NULL) ||
FmgrHookIsNeeded(functionId))
{
*mod = NULL; /* core binary */
diff --git a/src/backend/utils/fmgr/funcapi.c b/src/backend/utils/fmgr/funcapi.c
index 20f60392afe..d5984b415ed 100644
--- a/src/backend/utils/fmgr/funcapi.c
+++ b/src/backend/utils/fmgr/funcapi.c
@@ -1091,8 +1091,8 @@ get_func_result_name(Oid functionId)
elog(ERROR, "cache lookup failed for function %u", functionId);
/* If there are no named OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargnames))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargnames, NULL))
result = NULL;
else
{
@@ -1186,8 +1186,8 @@ build_function_result_tupdesc_t(HeapTuple procTuple)
return NULL;
/* If there are no OUT parameters, return NULL */
- if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes) ||
- heap_attisnull(procTuple, Anum_pg_proc_proargmodes))
+ if (heap_attisnull(procTuple, Anum_pg_proc_proallargtypes, NULL) ||
+ heap_attisnull(procTuple, Anum_pg_proc_proargmodes, NULL))
return NULL;
/* Get the data out of the tuple */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 3616a17b6fa..e2f16e39a67 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -799,7 +799,7 @@ extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
uint16 *infomask, bits8 *bit);
-extern bool heap_attisnull(HeapTuple tup, int attnum);
+extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
@@ -829,5 +829,8 @@ extern void heap_free_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple heap_copy_minimal_tuple(MinimalTuple mtup);
extern HeapTuple heap_tuple_from_minimal_tuple(MinimalTuple mtup);
extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
+extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
+
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97e..708160f645e 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -25,6 +25,8 @@ typedef struct attrDefault
char *adbin; /* nodeToString representation of expr */
} AttrDefault;
+typedef struct attrMissing *MissingPtr;
+
typedef struct constrCheck
{
char *ccname;
@@ -38,6 +40,7 @@ typedef struct tupleConstr
{
AttrDefault *defval; /* array */
ConstrCheck *check; /* array */
+ MissingPtr missing; /* missing attributes values, NULL if none */
uint16 num_defval;
uint16 num_check;
bool has_not_null;
diff --git a/src/include/access/tupdesc_details.h b/src/include/access/tupdesc_details.h
new file mode 100644
index 00000000000..741e996b3cc
--- /dev/null
+++ b/src/include/access/tupdesc_details.h
@@ -0,0 +1,29 @@
+/*-------------------------------------------------------------------------
+ *
+ * tupdesc_details.h
+ * POSTGRES tuple descriptor definitions we can't include everywhere
+ *
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/access/tupdesc_details.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TUPDESC_DETAILS_H
+#define TUPDESC_DETAILS_H
+
+/*
+ * Structure used to represent value to be used when the attribute is not
+ * present at all in a tuple, i.e. when the column was created after the tuple
+ */
+
+typedef struct attrMissing
+{
+ bool ammissingPresent; /* true if non-NULL missing value exists */
+ Datum ammissing; /* value when attribute is missing */
+} AttrMissing;
+
+#endif /* TUPDESC_DETAILS_H */
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 3308fa3dfd5..59fc0524947 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -23,6 +23,7 @@ typedef struct RawColumnDefault
{
AttrNumber attnum; /* attribute to attach default to */
Node *raw_default; /* default value (untransformed parse tree) */
+ bool missingMode; /* true if part of add column processing */
} RawColumnDefault;
typedef struct CookedConstraint
@@ -103,8 +104,11 @@ extern List *AddRelationNewConstraints(Relation rel,
bool is_local,
bool is_internal);
+extern void RelationClearMissing(Relation rel);
+
extern Oid StoreAttrDefault(Relation rel, AttrNumber attnum,
- Node *expr, bool is_internal);
+ Node *expr, bool is_internal,
+ bool add_column_mode);
extern Node *cookDefault(ParseState *pstate,
Node *raw_default,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834d..280e36c5d23 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -133,6 +133,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Has DEFAULT value or not */
bool atthasdef BKI_DEFAULT(f);
+ /* Has a missing value or not */
+ bool atthasmissing BKI_DEFAULT(f);
+
/* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */
char attidentity BKI_DEFAULT("");
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
@@ -191,11 +200,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
+#define Natts_pg_attribute 24
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
@@ -206,15 +215,16 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attalign 12
#define Anum_pg_attribute_attnotnull 13
#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
-
+#define Anum_pg_attribute_atthasmissing 15
+#define Anum_pg_attribute_attidentity 16
+#define Anum_pg_attribute_attisdropped 17
+#define Anum_pg_attribute_attislocal 18
+#define Anum_pg_attribute_attinhcount 19
+#define Anum_pg_attribute_attcollation 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
+#define Anum_pg_attribute_attmissingval 24
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 85cdb99f1f6..135f33d0f30 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/test/regress/expected/event_trigger.out b/src/test/regress/expected/event_trigger.out
index 88c68030815..008e859d4c2 100644
--- a/src/test/regress/expected/event_trigger.out
+++ b/src/test/regress/expected/event_trigger.out
@@ -389,8 +389,6 @@ alter table rewriteme alter column foo type numeric;
ERROR: rewrites not allowed
CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
alter table rewriteme add column baz int default 0;
-ERROR: rewrites not allowed
-CONTEXT: PL/pgSQL function test_evtrig_no_rewrite() line 3 at RAISE
-- test with more than one reason to rewrite a single table
CREATE OR REPLACE FUNCTION test_evtrig_no_rewrite() RETURNS event_trigger
LANGUAGE plpgsql AS $$
@@ -404,7 +402,7 @@ alter table rewriteme
add column onemore int default 0,
add column another int default -1,
alter column foo type numeric(10,4);
-NOTICE: Table 'rewriteme' is being rewritten (reason = 6)
+NOTICE: Table 'rewriteme' is being rewritten (reason = 4)
-- shouldn't trigger a table_rewrite event
alter table rewriteme alter column foo type numeric(12,4);
-- typed tables are rewritten when their type changes. Don't emit table
diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out
new file mode 100644
index 00000000000..9e74068a0a3
--- /dev/null
+++ b/src/test/regress/expected/fast_default.out
@@ -0,0 +1,515 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+NOTICE: rewriting table has_volatile for reason 2
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+INSERT INTO T VALUES (15), (16);
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+INSERT INTO T VALUES (17), (18);
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+INSERT INTO T VALUES (19), (20);
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+INSERT INTO T VALUES (21), (22);
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+INSERT INTO T VALUES (23), (24);
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+INSERT INTO T VALUES (25), (26);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+INSERT INTO T VALUES (27), (28);
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_timestamp_null | c_array | c_small | c_small_null | c_big | c_num | c_time | c_interval | c_hugetext_origdef | c_hugetext_newdef
+----+-------+----------+--------+------------+--------------------------+--------------------------+--------------------------+---------+--------------+-------------------+-------------------+----------+------------+--------------------+-------------------
+ 1 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 2 | 1 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 3 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 4 | 2 | hello | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 5 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 6 | 2 | dog | world | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 7 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 8 | 2 | dog | cat | 06-02-2016 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 9 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 10 | 2 | dog | cat | 01-01-2010 | Thu Sep 01 12:00:00 2016 | | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 11 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 12 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,the,real,world} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 13 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 14 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | -5 | | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 15 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 16 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | 180000000000018 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 17 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 18 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 1.00000000001 | 12:00:00 | @ 1 day | t | f
+ 19 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 20 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 12:00:00 | @ 1 day | t | f
+ 21 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 22 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 1 day | t | f
+ 23 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 24 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | @ 3 hours | t | f
+ 25 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 26 | 2 | dog | cat | 01-01-2010 | Thu Dec 31 11:12:13 1970 | Thu Sep 29 12:00:00 2016 | {This,is,no,fantasy} | 9 | 13 | -9999999999999999 | 2.000000000000002 | 23:59:59 | | f | t
+ 27 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+ 28 | 2 | | | | | Thu Sep 29 12:00:00 2016 | | | 13 | | | | | |
+(28 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+INSERT INTO T VALUES (7), (8);
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+INSERT INTO T VALUES (9), (10);
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+INSERT INTO T VALUES (11), (12);
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+INSERT INTO T VALUES (13), (14);
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+INSERT INTO T VALUES (15), (16);
+SELECT * FROM T;
+ pk | c_int | c_bpchar | c_text | c_date | c_timestamp | c_array
+----+-------+----------+--------------+------------+--------------------------+-------------------------------
+ 1 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 2 | 6 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 3 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 4 | 8 | abcd | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 5 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 6 | 8 | abc | abcdef | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 7 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 8 | 8 | abc | abcdefghijkl | 06-12-2016 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 9 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 10 | 8 | abc | abcdefghijkl | 12-28-2009 | Sun Sep 11 00:00:00 2016 | {This,is,abcd,the,real,world}
+ 11 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 12 | 8 | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,abcd,the,real,world}
+ 13 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 14 | | abc | abcdefghijkl | 12-28-2009 | Sat Jan 30 00:00:00 1971 | {This,is,a,fantasy}
+ 15 | | | | | |
+ 16 | | | | | |
+(16 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION foo(INT);
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+INSERT INTO T VALUES (1);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+NOTICE: rewriting table t for reason 2
+SELECT comp();
+ comp
+-----------
+ Rewritten
+(1 row)
+
+DROP TABLE T;
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_bigint = '-1'::integer)
+(5 rows)
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ c_bigint | c_text
+----------+--------
+ -1 | hello
+(1 row)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+ QUERY PLAN
+--------------------------------------------
+ Limit
+ Output: c_bigint, c_text
+ -> Seq Scan on fast_default.t
+ Output: c_bigint, c_text
+ Filter: (t.c_text = 'hello'::text)
+(5 rows)
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+ coalesce | coalesce
+----------+----------
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+ -1 | hello
+(10 rows)
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+ sum | max | min
+-----+-------+-----
+ 200 | hello | 31
+(1 row)
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 1 | -1 | hello
+ 2 | -1 | hello
+ 3 | -1 | hello
+ 4 | -1 | hello
+ 5 | -1 | hello
+ 6 | -1 | hello
+ 7 | -1 | hello
+ 8 | -1 | hello
+ 9 | -1 | hello
+ 10 | -1 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+(7 rows)
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ pk | c_bigint | c_text
+----+----------+--------
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(10 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+ QUERY PLAN
+----------------------------------------------------
+ Limit
+ Output: pk, c_bigint, c_text
+ -> Sort
+ Output: pk, c_bigint, c_text
+ Sort Key: t.c_bigint, t.c_text, t.pk
+ -> Seq Scan on fast_default.t
+ Output: pk, c_bigint, c_text
+ Filter: (t.c_bigint > '-1'::integer)
+(8 rows)
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ pk | c_bigint | c_text
+----+----------+--------
+ 10 | -1 | hello
+ 11 | 1 | hello
+ 12 | 2 | hello
+ 13 | 3 | hello
+ 14 | 4 | hello
+ 15 | 5 | hello
+ 16 | 6 | hello
+ 17 | 7 | hello
+ 18 | 8 | hello
+ 19 | 9 | hello
+ 20 | 10 | hello
+(11 rows)
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+ QUERY PLAN
+-----------------------------------------------------------
+ Delete on fast_default.t
+ Output: pk, c_bigint, c_text
+ -> Bitmap Heap Scan on fast_default.t
+ Output: ctid
+ Recheck Cond: ((t.pk >= 10) AND (t.pk <= 20))
+ -> Bitmap Index Scan on t_pkey
+ Index Cond: ((t.pk >= 10) AND (t.pk <= 20))
+(7 rows)
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+ pk | c_bigint | c_text
+----+----------+---------
+ 1 | -1 | "hello"
+ 2 | -1 | "hello"
+ 3 | -1 | "hello"
+ 4 | -1 | "hello"
+ 5 | -1 | "hello"
+ 6 | -1 | "hello"
+ 7 | -1 | "hello"
+ 8 | -1 | "hello"
+ 9 | -1 | "hello"
+(9 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+SELECT set('t');
+ set
+-----
+
+(1 row)
+
+INSERT INTO T VALUES (1), (2);
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+INSERT INTO T VALUES (3), (4);
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+INSERT INTO T VALUES (5), (6);
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+INSERT INTO T VALUES (7), (8);
+SELECT * FROM T ORDER BY pk;
+ pk | c_int | c_text
+----+-------+--------
+ 1 | -1 | Hello
+ 2 | -1 | Hello
+ 3 | -1 | Hello
+ 4 | -1 | Hello
+ 5 | -1 | Hello
+ 6 | -1 | Hello
+ 7 | 1 | world
+ 8 | 1 | world
+(8 rows)
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+SELECT c_text FROM T WHERE c_int = -1;
+ c_text
+--------
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+ Hello
+(6 rows)
+
+SELECT comp();
+ comp
+-----------
+ Unchanged
+(1 row)
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index d308a05117d..f4d77175548 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,7 +116,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare without_oid c
# ----------
# Another group of parallel tests
# ----------
-test: identity partition_join partition_prune reloptions hash_part indexing partition_aggregate
+test: identity partition_join partition_prune reloptions hash_part indexing partition_aggregate fast_default
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 45147e9328a..b54e30256ed 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -188,3 +188,4 @@ test: indexing
test: partition_aggregate
test: event_trigger
test: stats
+test: fast_default
diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql
new file mode 100644
index 00000000000..6a3f9b367e4
--- /dev/null
+++ b/src/test/regress/sql/fast_default.sql
@@ -0,0 +1,357 @@
+--
+-- ALTER TABLE ADD COLUMN DEFAULT test
+--
+
+SET search_path = fast_default;
+CREATE SCHEMA fast_default;
+CREATE TABLE m(id OID);
+INSERT INTO m VALUES (NULL::OID);
+
+CREATE FUNCTION set(tabname name) RETURNS VOID
+AS $$
+BEGIN
+ UPDATE m
+ SET id = (SELECT c.relfilenode
+ FROM pg_class AS c, pg_namespace AS s
+ WHERE c.relname = tabname
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION comp() RETURNS TEXT
+AS $$
+BEGIN
+ RETURN (SELECT CASE
+ WHEN m.id = c.relfilenode THEN 'Unchanged'
+ ELSE 'Rewritten'
+ END
+ FROM m, pg_class AS c, pg_namespace AS s
+ WHERE c.relname = 't'
+ AND c.relnamespace = s.oid
+ AND s.nspname = 'fast_default');
+END;
+$$ LANGUAGE 'plpgsql';
+
+CREATE FUNCTION log_rewrite() RETURNS event_trigger
+LANGUAGE plpgsql as
+$func$
+
+declare
+ this_schema text;
+begin
+ select into this_schema relnamespace::regnamespace::text
+ from pg_class
+ where oid = pg_event_trigger_table_rewrite_oid();
+ if this_schema = 'fast_default'
+ then
+ RAISE NOTICE 'rewriting table % for reason %',
+ pg_event_trigger_table_rewrite_oid()::regclass,
+ pg_event_trigger_table_rewrite_reason();
+ end if;
+end;
+$func$;
+
+CREATE TABLE has_volatile AS
+SELECT * FROM generate_series(1,10) id;
+
+
+CREATE EVENT TRIGGER has_volatile_rewrite
+ ON table_rewrite
+ EXECUTE PROCEDURE log_rewrite();
+
+-- only the last of these should trigger a rewrite
+ALTER TABLE has_volatile ADD col1 int;
+ALTER TABLE has_volatile ADD col2 int DEFAULT 1;
+ALTER TABLE has_volatile ADD col3 timestamptz DEFAULT current_timestamp;
+ALTER TABLE has_volatile ADD col4 int DEFAULT (random() * 10000)::int;
+
+
+
+-- Test a large sample of different datatypes
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT 'hello',
+ ALTER COLUMN c_int SET DEFAULT 2;
+
+INSERT INTO T VALUES (3), (4);
+
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'world',
+ ALTER COLUMN c_bpchar SET DEFAULT 'dog';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE DEFAULT '2016-06-02',
+ ALTER COLUMN c_text SET DEFAULT 'cat';
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP DEFAULT '2016-09-01 12:00:00',
+ ADD COLUMN c_timestamp_null TIMESTAMP,
+ ALTER COLUMN c_date SET DEFAULT '2010-01-01';
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT '{"This", "is", "the", "real", "world"}',
+ ALTER COLUMN c_timestamp SET DEFAULT '1970-12-31 11:12:13',
+ ALTER COLUMN c_timestamp_null SET DEFAULT '2016-09-29 12:00:00';
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ADD COLUMN c_small SMALLINT DEFAULT -5,
+ ADD COLUMN c_small_null SMALLINT,
+ ALTER COLUMN c_array
+ SET DEFAULT '{"This", "is", "no", "fantasy"}';
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ADD COLUMN c_big BIGINT DEFAULT 180000000000018,
+ ALTER COLUMN c_small SET DEFAULT 9,
+ ALTER COLUMN c_small_null SET DEFAULT 13;
+
+INSERT INTO T VALUES (15), (16);
+
+ALTER TABLE T ADD COLUMN c_num NUMERIC DEFAULT 1.00000000001,
+ ALTER COLUMN c_big SET DEFAULT -9999999999999999;
+
+INSERT INTO T VALUES (17), (18);
+
+ALTER TABLE T ADD COLUMN c_time TIME DEFAULT '12:00:00',
+ ALTER COLUMN c_num SET DEFAULT 2.000000000000002;
+
+INSERT INTO T VALUES (19), (20);
+
+ALTER TABLE T ADD COLUMN c_interval INTERVAL DEFAULT '1 day',
+ ALTER COLUMN c_time SET DEFAULT '23:59:59';
+
+INSERT INTO T VALUES (21), (22);
+
+ALTER TABLE T ADD COLUMN c_hugetext TEXT DEFAULT repeat('abcdefg',1000),
+ ALTER COLUMN c_interval SET DEFAULT '3 hours';
+
+INSERT INTO T VALUES (23), (24);
+
+ALTER TABLE T ALTER COLUMN c_interval DROP DEFAULT,
+ ALTER COLUMN c_hugetext SET DEFAULT repeat('poiuyt', 1000);
+
+INSERT INTO T VALUES (25), (26);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT,
+ ALTER COLUMN c_small DROP DEFAULT,
+ ALTER COLUMN c_big DROP DEFAULT,
+ ALTER COLUMN c_num DROP DEFAULT,
+ ALTER COLUMN c_time DROP DEFAULT,
+ ALTER COLUMN c_hugetext DROP DEFAULT;
+
+INSERT INTO T VALUES (27), (28);
+
+SELECT pk, c_int, c_bpchar, c_text, c_date, c_timestamp,
+ c_timestamp_null, c_array, c_small, c_small_null,
+ c_big, c_num, c_time, c_interval,
+ c_hugetext = repeat('abcdefg',1000) as c_hugetext_origdef,
+ c_hugetext = repeat('poiuyt', 1000) as c_hugetext_newdef
+FROM T ORDER BY pk;
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Test expressions in the defaults
+CREATE OR REPLACE FUNCTION foo(a INT) RETURNS TEXT AS $$
+DECLARE res TEXT := '';
+ i INT;
+BEGIN
+ i := 0;
+ WHILE (i < a) LOOP
+ res := res || chr(ascii('a') + i);
+ i := i + 1;
+ END LOOP;
+ RETURN res;
+END; $$ LANGUAGE PLPGSQL STABLE;
+
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT LENGTH(foo(6)));
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_bpchar BPCHAR(5) DEFAULT foo(4),
+ ALTER COLUMN c_int SET DEFAULT LENGTH(foo(8));
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT foo(6),
+ ALTER COLUMN c_bpchar SET DEFAULT foo(3);
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ADD COLUMN c_date DATE
+ DEFAULT '2016-06-02'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_text SET DEFAULT foo(12);
+
+INSERT INTO T VALUES (7), (8);
+
+ALTER TABLE T ADD COLUMN c_timestamp TIMESTAMP
+ DEFAULT '2016-09-01'::DATE + LENGTH(foo(10)),
+ ALTER COLUMN c_date
+ SET DEFAULT '2010-01-01'::DATE - LENGTH(foo(4));
+
+INSERT INTO T VALUES (9), (10);
+
+ALTER TABLE T ADD COLUMN c_array TEXT[]
+ DEFAULT ('{"This", "is", "' || foo(4) ||
+ '","the", "real", "world"}')::TEXT[],
+ ALTER COLUMN c_timestamp
+ SET DEFAULT '1970-12-31'::DATE + LENGTH(foo(30));
+
+INSERT INTO T VALUES (11), (12);
+
+ALTER TABLE T ALTER COLUMN c_int DROP DEFAULT,
+ ALTER COLUMN c_array
+ SET DEFAULT ('{"This", "is", "' || foo(1) ||
+ '", "fantasy"}')::text[];
+
+INSERT INTO T VALUES (13), (14);
+
+ALTER TABLE T ALTER COLUMN c_bpchar DROP DEFAULT,
+ ALTER COLUMN c_date DROP DEFAULT,
+ ALTER COLUMN c_text DROP DEFAULT,
+ ALTER COLUMN c_timestamp DROP DEFAULT,
+ ALTER COLUMN c_array DROP DEFAULT;
+
+INSERT INTO T VALUES (15), (16);
+
+SELECT * FROM T;
+
+SELECT comp();
+
+DROP TABLE T;
+
+DROP FUNCTION foo(INT);
+
+-- Fall back to full rewrite for volatile expressions
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+INSERT INTO T VALUES (1);
+
+SELECT set('t');
+
+-- now() is stable, because it returns the transaction timestamp
+ALTER TABLE T ADD COLUMN c1 TIMESTAMP DEFAULT now();
+
+SELECT comp();
+
+-- clock_timestamp() is volatile
+ALTER TABLE T ADD COLUMN c2 TIMESTAMP DEFAULT clock_timestamp();
+
+SELECT comp();
+
+DROP TABLE T;
+
+-- Simple querie
+CREATE TABLE T (pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T SELECT * FROM generate_series(1, 10) a;
+
+ALTER TABLE T ADD COLUMN c_bigint BIGINT NOT NULL DEFAULT -1;
+
+INSERT INTO T SELECT b, b - 10 FROM generate_series(11, 20) a(b);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'hello';
+
+INSERT INTO T SELECT b, b - 10, (b + 10)::text FROM generate_series(21, 30) a(b);
+
+-- WHERE clause
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT c_bigint, c_text FROM T WHERE c_bigint = -1 LIMIT 1;
+
+SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE) SELECT c_bigint, c_text FROM T WHERE c_text = 'hello' LIMIT 1;
+
+
+-- COALESCE
+SELECT COALESCE(c_bigint, pk), COALESCE(c_text, pk::text)
+FROM T
+ORDER BY pk LIMIT 10;
+
+-- Aggregate function
+SELECT SUM(c_bigint), MAX(c_text), MIN(c_text) FROM T;
+
+-- ORDER BY
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- LIMIT
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+SELECT * FROM T WHERE c_bigint > -1 ORDER BY c_bigint, c_text, pk LIMIT 10;
+
+-- DELETE with RETURNING
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+EXPLAIN (VERBOSE TRUE, COSTS FALSE)
+DELETE FROM T WHERE pk BETWEEN 10 AND 20 RETURNING *;
+
+-- UPDATE
+UPDATE T SET c_text = '"' || c_text || '"' WHERE pk < 10;
+SELECT * FROM T WHERE c_text LIKE '"%"' ORDER BY PK;
+
+SELECT comp();
+
+DROP TABLE T;
+
+
+-- Combine with other DDL
+CREATE TABLE T(pk INT NOT NULL PRIMARY KEY);
+
+SELECT set('t');
+
+INSERT INTO T VALUES (1), (2);
+
+ALTER TABLE T ADD COLUMN c_int INT NOT NULL DEFAULT -1;
+
+INSERT INTO T VALUES (3), (4);
+
+ALTER TABLE T ADD COLUMN c_text TEXT DEFAULT 'Hello';
+
+INSERT INTO T VALUES (5), (6);
+
+ALTER TABLE T ALTER COLUMN c_text SET DEFAULT 'world',
+ ALTER COLUMN c_int SET DEFAULT 1;
+
+INSERT INTO T VALUES (7), (8);
+
+SELECT * FROM T ORDER BY pk;
+
+-- Add an index
+CREATE INDEX i ON T(c_int, c_text);
+
+SELECT c_text FROM T WHERE c_int = -1;
+
+SELECT comp();
+
+DROP TABLE T;
+DROP FUNCTION set(name);
+DROP FUNCTION comp();
+DROP TABLE m;
+DROP TABLE has_volatile;
+DROP EVENT TRIGGER has_volatile_rewrite;
+DROP FUNCTION log_rewrite;
+DROP SCHEMA fast_default;
On 25 March 2018 at 20:09, David Rowley <david.rowley@2ndquadrant.com> wrote:
On 15 March 2018 at 21:33, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:rebased and mostly indented patch version attached.
Thanks. I've attached a version of this which applies, builds and
passes the regression tests on current master.Some conflicts were caused by 325f2ec555 and there was a new call to
heap_attisnull which needed to be updated.I'll look over this now.
I've attached a delta patch against the v17 patch that I attached
earlier. I didn't change much, but there did seem to be a few places
where the patch was not properly setting atthasmissing to false. Most
of the rest is just cosmetic stuff
With the attached applied, I'm happy to mark the patch as ready for
committer, however, Petr is also signed up to review, so will defer to
him to see if he has any comments before altering the commitfest app's
state.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Attachments:
fast_default-v17_fixes.patchapplication/octet-stream; name=fast_default-v17_fixes.patchDownload
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index ebdb120a1a8..d6a9d8c5808 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1155,8 +1155,9 @@
<entry></entry>
<entry>
This column has a value which is used where the column is entirely
- missing from the row, as happens when a column is added after the
- row is created. The actual value used is stored in the
+ missing from the row, as happens when a column is added with a
+ non-volatile <literal>DEFAULT</literal> value after the row is created.
+ The actual value used is stored in the
<structfield>attmissingval</structfield> column.
</entry>
</row>
@@ -1248,8 +1249,9 @@
<entry>
This column has a one element array containing the value used when the
column is entirely missing from the row, as happens when the column is
- added after the row is created. The value is only used when
- <structfield>atthasmissing</structfield> is true. If there is no value
+ added with a non-volatile <literal>DEFAULT</literal> value after the
+ row is created. The value is only used when
+ <structfield>atthasmissing</structfield> is true. If there is no value
the column is null.
</entry>
</row>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index edf459ae6b1..69f3355eded 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -1187,8 +1187,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
When a column is added with <literal>ADD COLUMN</literal> and a
non-volatile <literal>DEFAULT</literal> is specified, the default is
evaluated at the time of the statement and the result stored in the
- table's metadata. That value will be used for the column for all existing
- rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
+ table's metadata. That value will be used for the column for all existing
+ rows. If no <literal>DEFAULT</literal> is specified, NULL is used. In
neither case is a rewrite of the table required.
</para>
@@ -1201,10 +1201,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
type or an unconstrained domain over the new type, a table rewrite is not
needed; but any indexes on the affected columns must still be rebuilt.
Adding or removing a system <literal>oid</literal> column also requires
- rewriting the entire table.
- Table and/or index rebuilds may take a significant amount of time
- for a large table; and will temporarily require as much as double the disk
- space.
+ rewriting the entire table. Table and/or index rebuilds may take a
+ significant amount of time for a large table; and will temporarily require
+ as much as double the disk space.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 6d19537489c..aae7e2d9049 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -85,7 +85,6 @@ getmissingattr(TupleDesc tupleDesc,
int attnum, bool *isnull)
{
Form_pg_attribute att;
- AttrMissing *attrmiss;
Assert(attnum <= tupleDesc->natts);
Assert(attnum > 0);
@@ -94,6 +93,8 @@ getmissingattr(TupleDesc tupleDesc,
if (att->atthasmissing)
{
+ AttrMissing *attrmiss;
+
Assert(tupleDesc->constr);
Assert(tupleDesc->constr->missing);
@@ -323,9 +324,6 @@ fill_val(Form_pg_attribute att,
data += data_length;
*dataP = data;
-
-
-
}
/*
@@ -1558,7 +1556,8 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
elog(ERROR, "cannot extract attribute from empty tuple slot");
/*
- * return NULL if attnum is out of range according to the tuple
+ * return NULL or missing value if attnum is out of range according to the
+ * tuple
*
* (We have to check this separately because of various inheritance and
* table-alteration scenarios: the tuple could be either longer or shorter
@@ -1566,9 +1565,7 @@ slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull)
*/
tup = tuple->t_data;
if (attnum > HeapTupleHeaderGetNatts(tup))
- {
return getmissingattr(slot->tts_tupleDescriptor, attnum, isnull);
- }
/*
* check if target attribute is null: no point in groveling through tuple
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 54393607222..2658399484b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -131,6 +131,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
}
@@ -246,6 +247,7 @@ TupleDescCopy(TupleDesc dst, TupleDesc src)
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
}
dst->constr = NULL;
@@ -298,6 +300,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
/* since we're not copying constraints or defaults, clear these */
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
+ dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
}
@@ -697,6 +700,7 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
att->attnotnull = false;
att->atthasdef = false;
+ att->atthasmissing = false;
att->attidentity = '\0';
att->attisdropped = false;
att->attislocal = true;
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 2b34bb8313d..a1def77944c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -1934,14 +1934,13 @@ heap_drop_with_catalog(Oid relid)
/*
* RelationClearMissing
*
- * set atthasmissing and attmissingval to false/null for all attributes
+ * Set atthasmissing and attmissingval to false/null for all attributes
* where they are currently set. This can be safely and usefully done if
* the table is rewritten (e.g. by VACUUM FULL or CLUSTER) where we know there
* are no rows left with less than a full complement of attributes.
*
* The caller must have an AccessExclusive lock on the relation.
*/
-
void
RelationClearMissing(Relation rel)
{
@@ -1970,7 +1969,7 @@ RelationClearMissing(Relation rel)
/* Get a lock on pg_attribute */
attr_rel = heap_open(AttributeRelationId, RowExclusiveLock);
- /* process each non-system attribute */
+ /* process each non-system attribute, including any dropped columns */
for (attnum = 1; attnum <= natts; attnum++)
{
tuple = SearchSysCache2(ATTNUM,
@@ -2027,19 +2026,9 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
Relation attrrel;
HeapTuple atttup;
Form_pg_attribute attStruct;
- Form_pg_attribute defAttStruct;
Oid attrdefOid;
ObjectAddress colobject,
defobject;
- ExprState *exprState;
- Expr *expr2 = (Expr *) expr;
- EState *estate = NULL;
- ExprContext *econtext;
- Datum valuesAtt[Natts_pg_attribute];
- bool nullsAtt[Natts_pg_attribute];
- bool replacesAtt[Natts_pg_attribute];
- Datum missingval = (Datum) 0;
- bool missingIsNull = true;
/*
* Flatten expression to string form for storage.
@@ -2094,6 +2083,18 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
attStruct = (Form_pg_attribute) GETSTRUCT(atttup);
if (!attStruct->atthasdef)
{
+ Form_pg_attribute defAttStruct;
+
+ ExprState *exprState;
+ Expr *expr2 = (Expr *) expr;
+ EState *estate = NULL;
+ ExprContext *econtext;
+ Datum valuesAtt[Natts_pg_attribute];
+ bool nullsAtt[Natts_pg_attribute];
+ bool replacesAtt[Natts_pg_attribute];
+ Datum missingval = (Datum) 0;
+ bool missingIsNull = true;
+
MemSet(valuesAtt, 0, sizeof(valuesAtt));
MemSet(nullsAtt, false, sizeof(nullsAtt));
MemSet(replacesAtt, false, sizeof(replacesAtt));
@@ -2110,6 +2111,8 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
missingval = ExecEvalExpr(exprState, econtext,
&missingIsNull);
+ FreeExecutorState(estate);
+
defAttStruct = TupleDescAttr(rel->rd_att, attnum - 1);
if (missingIsNull)
@@ -2139,6 +2142,10 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
valuesAtt, nullsAtt, replacesAtt);
CatalogTupleUpdate(attrrel, &atttup->t_self, atttup);
+
+ if (!missingIsNull)
+ pfree(DatumGetPointer(missingval));
+
}
heap_close(attrrel, RowExclusiveLock);
heap_freetuple(atttup);
@@ -2169,14 +2176,6 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
InvokeObjectPostCreateHookArg(AttrDefaultRelationId,
RelationGetRelid(rel), attnum, is_internal);
- if (estate)
- {
- FreeExecutorState(estate);
- }
-
- if (!missingIsNull)
- pfree(DatumGetPointer(missingval));
-
return attrdefOid;
}
@@ -2446,11 +2445,9 @@ AddRelationNewConstraints(Relation rel,
(IsA(expr, Const) &&((Const *) expr)->constisnull))
continue;
- /* If the default is volatile we cannot use a missing value */
+ /* If the DEFAULT is volatile we cannot use a missing value */
if (colDef->missingMode && contain_volatile_functions((Node *) expr))
- {
colDef->missingMode = false;
- }
defOid = StoreAttrDefault(rel, colDef->attnum, expr, is_internal,
colDef->missingMode);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c0790a724f8..83a881eff38 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5449,6 +5449,12 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rawEnt = (RawColumnDefault *) palloc(sizeof(RawColumnDefault));
rawEnt->attnum = attribute.attnum;
rawEnt->raw_default = copyObject(colDef->raw_default);
+
+ /*
+ * Attempt to skip a complete table rewrite by storing the specified
+ * DEFAULT value outside of the heap. This may be disabled inside
+ * AddRelationNewConstraints if the optimization cannot be applied.
+ */
rawEnt->missingMode = true;
/*
@@ -5466,9 +5472,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
* a rewrite
*/
if (!rawEnt->missingMode)
- {
tab->rewrite |= AT_REWRITE_DEFAULT_VAL;
- }
}
/*
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 5348c77b23d..0231f8bf7c6 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -1494,11 +1494,11 @@ relation_excluded_by_constraints(PlannerInfo *root,
* step at runtime, so we use such tlists preferentially for scan nodes.
*
* Exception: if there are any dropped or missing columns, we punt and return
- * NIL. Ideally we would like to handle these cases too. However
- * this creates problems for ExecTypeFromTL, which may be asked to build a
- * tupdesc for a tlist that includes vars of no-longer-existent types. In
- * theory we could dig out the required info from the pg_attribute entries of
- * the relation, but that data is not readily available to ExecTypeFromTL.
+ * NIL. Ideally we would like to handle these cases too. However this
+ * creates problems for ExecTypeFromTL, which may be asked to build a tupdesc
+ * for a tlist that includes vars of no-longer-existent types. In theory we
+ * could dig out the required info from the pg_attribute entries of the
+ * relation, but that data is not readily available to ExecTypeFromTL.
* For now, we don't apply the physical-tlist optimization when there are
* dropped cols.
*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index db6d8b0ceb3..555ffed43ff 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -680,9 +680,8 @@ RelationBuildTupleDesc(Relation relation)
AttrDefaultFetch(relation);
}
else
- {
constr->num_defval = 0;
- }
+
constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index e2f16e39a67..002c104ce3c 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -832,5 +832,4 @@ extern MinimalTuple minimal_tuple_from_heap_tuple(HeapTuple htup);
extern HeapTuple heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
extern MinimalTuple minimal_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc);
-
#endif /* HTUP_DETAILS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 280e36c5d23..5bb64f7c31d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -204,7 +204,7 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attstattarget 4
#define Anum_pg_attribute_attlen 5
#define Anum_pg_attribute_attnum 6
#define Anum_pg_attribute_attndims 7
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index b54e30256ed..fccddc6ecd4 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -186,6 +186,6 @@ test: reloptions
test: hash_part
test: indexing
test: partition_aggregate
+test: fast_default
test: event_trigger
test: stats
-test: fast_default
On Sun, Mar 25, 2018 at 9:13 PM, David Rowley
<david.rowley@2ndquadrant.com> wrote:
On 25 March 2018 at 20:09, David Rowley <david.rowley@2ndquadrant.com> wrote:
On 15 March 2018 at 21:33, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:rebased and mostly indented patch version attached.
Thanks. I've attached a version of this which applies, builds and
passes the regression tests on current master.Some conflicts were caused by 325f2ec555 and there was a new call to
heap_attisnull which needed to be updated.I'll look over this now.
I've attached a delta patch against the v17 patch that I attached
earlier. I didn't change much, but there did seem to be a few places
where the patch was not properly setting atthasmissing to false. Most
of the rest is just cosmetic stuffWith the attached applied, I'm happy to mark the patch as ready for
committer, however, Petr is also signed up to review, so will defer to
him to see if he has any comments before altering the commitfest app's
state.
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On 25 March 2018 at 23:43, David Rowley <david.rowley@2ndquadrant.com> wrote:
With the attached applied, I'm happy to mark the patch as ready for
committer, however, Petr is also signed up to review, so will defer to
him to see if he has any comments before altering the commitfest app's
state.
Petr won't have time to look so I'll adjust the commitfest app's state now.
--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Hi,
On 2018-03-26 09:02:10 +1030, Andrew Dunstan wrote:
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.
Here's a bit of post commit review:
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/*
* If tuple doesn't have all the atts indicated by tupleDesc, read the
- * rest as null
+ * rest as NULLs or missing values
*/
- for (; attno < attnum; attno++)
- {
- slot->tts_values[attno] = (Datum) 0;
- slot->tts_isnull[attno] = true;
- }
+ if (attno < attnum)
+ slot_getmissingattrs(slot, attno, attnum);
+
slot->tts_nvalid = attnum;
}
It's worthwhile to note that this'll re-process *all* missing values,
even if they've already been deformed. I.e. if
slot_getmissingattrs(.., first-missing)
slot_getmissingattrs(.., first-missing + 1)
slot_getmissingattrs(.., first-missing + 2)
is called, all three missing values will be copied every time. That's
because tts_nvalid isn't taken into account. I wonder if slot_getmissingattrs
could take tts_nvalid into account?
I also wonder if this doesn't deserve an unlikely(), to avoid the cost
of spilling registers in the hot branch of not missing any values.
+ else
+ {
+ /* if there is a missing values array we must process them one by one */
+ for (missattnum = lastAttNum - 1;
+ missattnum >= startAttNum;
+ missattnum--)
+ {
+ slot->tts_values[missattnum] = attrmiss[missattnum].ammissing;
+ slot->tts_isnull[missattnum] =
+ !attrmiss[missattnum].ammissingPresent;
+ }
+ }
+}
Why is this done backwards? It's noticeably slower to walk arrays
backwards on some CPU microarchitectures.
@@ -176,6 +179,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}
@@ -469,6 +503,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
}
+ if (constr1->missing)
+ {
+ if (!constr2->missing)
+ return false;
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttrMissing *missval1 = constr1->missing + i;
+ AttrMissing *missval2 = constr2->missing + i;
It's a bit odd to not use array indexing here?
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot)
if (slot->tts_mintuple)
return heap_copy_minimal_tuple(slot->tts_mintuple);
if (slot->tts_tuple)
- return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ {
+ if (TTS_HAS_PHYSICAL_TUPLE(slot) &&
+ HeapTupleHeaderGetNatts(slot->tts_tuple->t_data)
+ < slot->tts_tupleDescriptor->natts)
+ return minimal_expand_tuple(slot->tts_tuple,
+ slot->tts_tupleDescriptor);
+ else
+ return minimal_tuple_from_heap_tuple(slot->tts_tuple);
+ }
What's the TTS_HAS_PHYSICAL_TUPLE doing here? How can this be relevant
given the previous tts_mintuple check? Am I missing something?
@@ -563,10 +569,63 @@ RelationBuildTupleDesc(Relation relation)
MemoryContextAllocZero(CacheMemoryContext,
relation->rd_rel->relnatts *
sizeof(AttrDefault));
- attrdef[ndef].adnum = attp->attnum;
+ attrdef[ndef].adnum = attnum;
attrdef[ndef].adbin = NULL;
+
ndef++;
}
+
+ /* Likewise for a missing value */
+ if (attp->atthasmissing)
+ {
+ Datum missingval;
+ bool missingNull;
+
+ /* Do we have a missing value? */
+ missingval = heap_getattr(pg_attribute_tuple,
+ Anum_pg_attribute_attmissingval,
+ pg_attribute_desc->rd_att,
+ &missingNull);
+ if (!missingNull)
+ {
+ /* Yes, fetch from the array */
+ MemoryContext oldcxt;
+ bool is_null;
+ int one = 1;
+ Datum missval;
+
+ if (attrmiss == NULL)
+ attrmiss = (AttrMissing *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ relation->rd_rel->relnatts *
+ sizeof(AttrMissing));
+
+ missval = array_get_element(missingval,
+ 1,
+ &one,
+ -1,
+ attp->attlen,
+ attp->attbyval,
+ attp->attalign,
+ &is_null);
+ Assert(!is_null);
+ if (attp->attbyval)
+ {
+ /* for copy by val just copy the datum direct */
+ attrmiss[attnum - 1].ammissing = missval;
+ }
+ else
+ {
+ /* otherwise copy in the correct context */
+ oldcxt = MemoryContextSwitchTo(CacheMemoryContext);
+ attrmiss[attnum - 1].ammissing = datumCopy(missval,
+ attp->attbyval,
+ attp->attlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ attrmiss[attnum - 1].ammissingPresent = true;
+ }
+ }
need--;
if (need == 0)
break;
I'm still strongly object to do doing this unconditionally for queries
that never need any of this. We're can end up with a significant number
of large tuples in memory here, and then copy that through dozens of
tupledesc copies in queries.
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+
+ /*
+ * Missing value for added columns. This is a one element array which lets
+ * us store a value of the attribute type here.
+ */
+ anyarray attmissingval BKI_DEFAULT(_null_);
#endif
} FormData_pg_attribute;
Still think this is a bad location, and it'll reduce cache hit ratio for
catalog lookups.
Greetings,
Andres Freund
On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-03-26 09:02:10 +1030, Andrew Dunstan wrote:
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.Here's a bit of post commit review:
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/* * If tuple doesn't have all the atts indicated by tupleDesc, read the - * rest as null + * rest as NULLs or missing values */ - for (; attno < attnum; attno++) - { - slot->tts_values[attno] = (Datum) 0; - slot->tts_isnull[attno] = true; - } + if (attno < attnum) + slot_getmissingattrs(slot, attno, attnum); + slot->tts_nvalid = attnum; }It's worthwhile to note that this'll re-process *all* missing values,
even if they've already been deformed. I.e. if
slot_getmissingattrs(.., first-missing)
slot_getmissingattrs(.., first-missing + 1)
slot_getmissingattrs(.., first-missing + 2)
is called, all three missing values will be copied every time. That's
because tts_nvalid isn't taken into account. I wonder if slot_getmissingattrs
could take tts_nvalid into account?I also wonder if this doesn't deserve an unlikely(), to avoid the cost
of spilling registers in the hot branch of not missing any values.
One of us at least is very confused about this function.
slot_getmissingattrs() gets called at most once by slot_getsomeattrs
and never for any attribute that's already been deformed.
+ else + { + /* if there is a missing values array we must process them one by one */ + for (missattnum = lastAttNum - 1; + missattnum >= startAttNum; + missattnum--) + { + slot->tts_values[missattnum] = attrmiss[missattnum].ammissing; + slot->tts_isnull[missattnum] = + !attrmiss[missattnum].ammissingPresent; + } + } +}Why is this done backwards? It's noticeably slower to walk arrays
backwards on some CPU microarchitectures.
I'll change it.
@@ -176,6 +179,23 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
}
}@@ -469,6 +503,29 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2) if (strcmp(defval1->adbin, defval2->adbin) != 0) return false; } + if (constr1->missing) + { + if (!constr2->missing) + return false; + for (i = 0; i < tupdesc1->natts; i++) + { + AttrMissing *missval1 = constr1->missing + i; + AttrMissing *missval2 = constr2->missing + i;It's a bit odd to not use array indexing here?
*shrug* Maybe. I'll change it if you like, doesn't seem that important or odd.
@@ -625,7 +625,15 @@ ExecCopySlotMinimalTuple(TupleTableSlot *slot) if (slot->tts_mintuple) return heap_copy_minimal_tuple(slot->tts_mintuple); if (slot->tts_tuple) - return minimal_tuple_from_heap_tuple(slot->tts_tuple); + { + if (TTS_HAS_PHYSICAL_TUPLE(slot) && + HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) + < slot->tts_tupleDescriptor->natts) + return minimal_expand_tuple(slot->tts_tuple, + slot->tts_tupleDescriptor); + else + return minimal_tuple_from_heap_tuple(slot->tts_tuple); + }What's the TTS_HAS_PHYSICAL_TUPLE doing here? How can this be relevant
given the previous tts_mintuple check? Am I missing something?
Hmm, that dates back to the original patch. My bad, I should have
picked that up. I'll just remove it.
@@ -563,10 +569,63 @@ RelationBuildTupleDesc(Relation relation) MemoryContextAllocZero(CacheMemoryContext, relation->rd_rel->relnatts * sizeof(AttrDefault)); - attrdef[ndef].adnum = attp->attnum; + attrdef[ndef].adnum = attnum; attrdef[ndef].adbin = NULL; + ndef++; } + + /* Likewise for a missing value */ + if (attp->atthasmissing) + { + Datum missingval; + bool missingNull; + + /* Do we have a missing value? */ + missingval = heap_getattr(pg_attribute_tuple, + Anum_pg_attribute_attmissingval, + pg_attribute_desc->rd_att, + &missingNull); + if (!missingNull) + { + /* Yes, fetch from the array */ + MemoryContext oldcxt; + bool is_null; + int one = 1; + Datum missval; + + if (attrmiss == NULL) + attrmiss = (AttrMissing *) + MemoryContextAllocZero(CacheMemoryContext, + relation->rd_rel->relnatts * + sizeof(AttrMissing)); + + missval = array_get_element(missingval, + 1, + &one, + -1, + attp->attlen, + attp->attbyval, + attp->attalign, + &is_null); + Assert(!is_null); + if (attp->attbyval) + { + /* for copy by val just copy the datum direct */ + attrmiss[attnum - 1].ammissing = missval; + } + else + { + /* otherwise copy in the correct context */ + oldcxt = MemoryContextSwitchTo(CacheMemoryContext); + attrmiss[attnum - 1].ammissing = datumCopy(missval, + attp->attbyval, + attp->attlen); + MemoryContextSwitchTo(oldcxt); + } + attrmiss[attnum - 1].ammissingPresent = true; + } + } need--; if (need == 0) break;I'm still strongly object to do doing this unconditionally for queries
that never need any of this. We're can end up with a significant number
of large tuples in memory here, and then copy that through dozens of
tupledesc copies in queries.
We're just doing the same thing we do for default values.
None of the tests I did with large numbers of missing values seemed to
show performance impacts of the kind you describe. Now, none of the
queries were particularly complex, but the worst case was from
actually using very large numbers of attributes with missing values,
where we'd need this data anyway. If we just selected a few attributes
performance went up rather than down.
@@ -167,6 +170,12 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* Column-level FDW options */ text attfdwoptions[1] BKI_DEFAULT(_null_); + + /* + * Missing value for added columns. This is a one element array which lets + * us store a value of the attribute type here. + */ + anyarray attmissingval BKI_DEFAULT(_null_); #endif } FormData_pg_attribute;Still think this is a bad location, and it'll reduce cache hit ratio for
catalog lookups.
As I think I mentioned before, this was discussed previously and as I
understood it this was the consensus location for it.
pg_attrdef isn't really a good place for it (what if they drop the
default?). So the only alternative then would be a completely new
catalog. I'd need a deal of convincing that that was justified.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund <andres@anarazel.de> wrote:
+ /* + * Missing value for added columns. This is a one element array which lets + * us store a value of the attribute type here. + */ + anyarray attmissingval BKI_DEFAULT(_null_); #endif } FormData_pg_attribute;Still think this is a bad location, and it'll reduce cache hit ratio for
catalog lookups.
As I think I mentioned before, this was discussed previously and as I
understood it this was the consensus location for it.
I don't have a problem with putting that in pg_attribute (and I certainly
agree with not putting it in pg_attrdef). But "anyarray" seems like a
damn strange, and bulky, choice. Why not just make it a bytea holding the
bits for the value, nothing more?
regards, tom lane
On Thu, Mar 29, 2018 at 10:19 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund <andres@anarazel.de> wrote:
+ /* + * Missing value for added columns. This is a one element array which lets + * us store a value of the attribute type here. + */ + anyarray attmissingval BKI_DEFAULT(_null_); #endif } FormData_pg_attribute;Still think this is a bad location, and it'll reduce cache hit ratio for
catalog lookups.As I think I mentioned before, this was discussed previously and as I
understood it this was the consensus location for it.I don't have a problem with putting that in pg_attribute (and I certainly
agree with not putting it in pg_attrdef). But "anyarray" seems like a
damn strange, and bulky, choice. Why not just make it a bytea holding the
bits for the value, nothing more?
That's what we started with and Andres suggested we change it. One
virtue of the array is that is displays nicely when examining
pg_attribute. But changing it back would be easy enough.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Mon, Mar 26, 2018 at 9:32 AM, Andrew Dunstan <andrew.dunstan@2ndquadrant.
com> wrote:
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.
I have some comments with the committed patch.
@@ -663,7 +671,23 @@ ExecFetchSlotTuple(TupleTableSlot *slot)
* If we have a regular physical tuple then just return it.
*/
if (TTS_HAS_PHYSICAL_TUPLE(slot))
- return slot->tts_tuple;
+ {
+ if (HeapTupleHeaderGetNatts(slot->tts_tuple->t_data) <
+ slot->tts_tupleDescriptor->natts)
+ {
+ MemoryContext oldContext =
MemoryContextSwitchTo(slot->tts_mcxt);
+
+ slot->tts_tuple = heap_expand_tuple(slot->tts_tuple,
+
slot->tts_tupleDescriptor);
+ slot->tts_shouldFree = true;
+ MemoryContextSwitchTo(oldContext);
+ return slot->tts_tuple;
+ }
+ else
+ {
+ return slot->tts_tuple;
+ }
+ }
In the above scenario, directly replacing the slot->tts_tuple without
freeing the exisitng
tuple will unnecessarily increase the slot context memory size, this may
lead to a problem
if the same slot is used for many tuples. Better to use ExecStoreTuple()
function to update
the slot tuple.
Regards,
Hari Babu
Fujitsu Australia
Hi,
On 2018-03-29 10:16:23 +1030, Andrew Dunstan wrote:
On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-03-26 09:02:10 +1030, Andrew Dunstan wrote:
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.Here's a bit of post commit review:
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/* * If tuple doesn't have all the atts indicated by tupleDesc, read the - * rest as null + * rest as NULLs or missing values */ - for (; attno < attnum; attno++) - { - slot->tts_values[attno] = (Datum) 0; - slot->tts_isnull[attno] = true; - } + if (attno < attnum) + slot_getmissingattrs(slot, attno, attnum); + slot->tts_nvalid = attnum; }It's worthwhile to note that this'll re-process *all* missing values,
even if they've already been deformed. I.e. if
slot_getmissingattrs(.., first-missing)
slot_getmissingattrs(.., first-missing + 1)
slot_getmissingattrs(.., first-missing + 2)
is called, all three missing values will be copied every time. That's
because tts_nvalid isn't taken into account. I wonder if slot_getmissingattrs
could take tts_nvalid into account?I also wonder if this doesn't deserve an unlikely(), to avoid the cost
of spilling registers in the hot branch of not missing any values.
One of us at least is very confused about this function.
slot_getmissingattrs() gets called at most once by slot_getsomeattrs
and never for any attribute that's already been deformed.
Imagine the same slot being used in two different expressions. The
typical case for that is e.g. something like
SELECT a,b,c,d FROM foo WHERE b = 1;
this'll trigger one slot_getsomeattrs(slot, 2) call from within qual
evaluation, and then a slot_getsomeattrs(slot, 4) from within the
projection code. If you then imagine a tuple where only the first
column exists physically, we'd copy b twice, because attno is only going
to be one 1. I think we might just want to check tts nvalid in
getmissingattrs?
I'm still strongly object to do doing this unconditionally for queries
that never need any of this. We're can end up with a significant number
of large tuples in memory here, and then copy that through dozens of
tupledesc copies in queries.
We're just doing the same thing we do for default values.
That's really not a defense. It's hurting us there too.
None of the tests I did with large numbers of missing values seemed to
show performance impacts of the kind you describe. Now, none of the
queries were particularly complex, but the worst case was from
actually using very large numbers of attributes with missing values,
where we'd need this data anyway. If we just selected a few attributes
performance went up rather than down.
Now imagine a partitioned workload with a few thousand partitions over a
few tables. The additional memory is going to start being noticeable.
pg_attrdef isn't really a good place for it (what if they drop the
default?). So the only alternative then would be a completely new
catalog. I'd need a deal of convincing that that was justified.
There's plenty databases with pg_attribute being many gigabytes large,
and this is going to make that even worse.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
There's plenty databases with pg_attribute being many gigabytes large,
and this is going to make that even worse.
Only if you imagine that a sizable fraction of the columns have fast
default values, which seems somewhat unlikely.
regards, tom lane
Hi,
On 2018-03-29 17:27:47 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
There's plenty databases with pg_attribute being many gigabytes large,
and this is going to make that even worse.Only if you imagine that a sizable fraction of the columns have fast
default values, which seems somewhat unlikely.
Why is that unlikely? In the field it's definitely not uncommon to
define default values for just about every column. And in a lot of cases
that'll mean we'll end up with pg_attribute containing default values
for most columns but the ones defined at table creation. A lot of
frameworks make it a habit to add columns near exclusively in
incremental steps. You'd only get rid of them if you force an operation
that does a full table rewrite, which often enough is impractical.
Greetings,
Andres Freund
On 03/29/2018 11:31 PM, Andres Freund wrote:
Hi,
On 2018-03-29 17:27:47 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
There's plenty databases with pg_attribute being many gigabytes large,
and this is going to make that even worse.Only if you imagine that a sizable fraction of the columns have fast
default values, which seems somewhat unlikely.Why is that unlikely? In the field it's definitely not uncommon to
define default values for just about every column. And in a lot of cases
that'll mean we'll end up with pg_attribute containing default values
for most columns but the ones defined at table creation. A lot of
frameworks make it a habit to add columns near exclusively in
incremental steps. You'd only get rid of them if you force an operation
that does a full table rewrite, which often enough is impractical.
I don't quite see how moving that gets solved by moving the info into a
different catalog? We would need to fetch it whenever attribute
meta-data from pg_attribute are loaded.
cheers
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On 2018-03-30 00:28:43 +0200, Tomas Vondra wrote:
Why is that unlikely? In the field it's definitely not uncommon to
define default values for just about every column. And in a lot of cases
that'll mean we'll end up with pg_attribute containing default values
for most columns but the ones defined at table creation. A lot of
frameworks make it a habit to add columns near exclusively in
incremental steps. You'd only get rid of them if you force an operation
that does a full table rewrite, which often enough is impractical.I don't quite see how moving that gets solved by moving the info into a
different catalog? We would need to fetch it whenever attribute
meta-data from pg_attribute are loaded.
I'm also complaining nearby that the "stored default" values should only
be loaded on demand. We very regularly build relcache entries that'll
never have their default values queried. Even moreso with partitioned
tables.
Greetings,
Andres Freund
On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-03-29 10:16:23 +1030, Andrew Dunstan wrote:
On Wed, Mar 28, 2018 at 5:30 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
On 2018-03-26 09:02:10 +1030, Andrew Dunstan wrote:
Thanks for this, all looks good. Here is the consolidate patch
rebased. If there are no further comments I propose to commit this in
a few days time.Here's a bit of post commit review:
@@ -1310,13 +1679,11 @@ slot_getsomeattrs(TupleTableSlot *slot, int attnum)
/* * If tuple doesn't have all the atts indicated by tupleDesc, read the - * rest as null + * rest as NULLs or missing values */ - for (; attno < attnum; attno++) - { - slot->tts_values[attno] = (Datum) 0; - slot->tts_isnull[attno] = true; - } + if (attno < attnum) + slot_getmissingattrs(slot, attno, attnum); + slot->tts_nvalid = attnum; }It's worthwhile to note that this'll re-process *all* missing values,
even if they've already been deformed. I.e. if
slot_getmissingattrs(.., first-missing)
slot_getmissingattrs(.., first-missing + 1)
slot_getmissingattrs(.., first-missing + 2)
is called, all three missing values will be copied every time. That's
because tts_nvalid isn't taken into account. I wonder if slot_getmissingattrs
could take tts_nvalid into account?I also wonder if this doesn't deserve an unlikely(), to avoid the cost
of spilling registers in the hot branch of not missing any values.One of us at least is very confused about this function.
slot_getmissingattrs() gets called at most once by slot_getsomeattrs
and never for any attribute that's already been deformed.Imagine the same slot being used in two different expressions. The
typical case for that is e.g. something like
SELECT a,b,c,d FROM foo WHERE b = 1;this'll trigger one slot_getsomeattrs(slot, 2) call from within qual
evaluation, and then a slot_getsomeattrs(slot, 4) from within the
projection code. If you then imagine a tuple where only the first
column exists physically, we'd copy b twice, because attno is only going
to be one 1. I think we might just want to check tts nvalid in
getmissingattrs?
OK. so add something like this to the top of slot_getmissingattrs()?
startAttNum = Max(startAttNum, slot->tts_nvalid);
I'm still strongly object to do doing this unconditionally for queries
that never need any of this. We're can end up with a significant number
of large tuples in memory here, and then copy that through dozens of
tupledesc copies in queries.We're just doing the same thing we do for default values.
That's really not a defense. It's hurting us there too.
I will take a look and see if it can be avoided easily.
None of the tests I did with large numbers of missing values seemed to
show performance impacts of the kind you describe. Now, none of the
queries were particularly complex, but the worst case was from
actually using very large numbers of attributes with missing values,
where we'd need this data anyway. If we just selected a few attributes
performance went up rather than down.Now imagine a partitioned workload with a few thousand partitions over a
few tables. The additional memory is going to start being noticeable.pg_attrdef isn't really a good place for it (what if they drop the
default?). So the only alternative then would be a completely new
catalog. I'd need a deal of convincing that that was justified.There's plenty databases with pg_attribute being many gigabytes large,
and this is going to make that even worse.
I'll change it if there's a consensus about it, but so far the only
other opinion has been from Tom who doesn't apparently see much of a
problem.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Hi,
On 2018-03-30 10:08:54 +1030, Andrew Dunstan wrote:
On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund <andres@anarazel.de> wrote:
this'll trigger one slot_getsomeattrs(slot, 2) call from within qual
evaluation, and then a slot_getsomeattrs(slot, 4) from within the
projection code. If you then imagine a tuple where only the first
column exists physically, we'd copy b twice, because attno is only going
to be one 1. I think we might just want to check tts nvalid in
getmissingattrs?
OK. so add something like this to the top of slot_getmissingattrs()?
startAttNum = Max(startAttNum, slot->tts_nvalid);
Yea, I think that should do the trick.
Greetings,
Andres Freund
On 2018-03-29 16:40:29 -0700, Andres Freund wrote:
Hi,
On 2018-03-30 10:08:54 +1030, Andrew Dunstan wrote:
On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund <andres@anarazel.de> wrote:
this'll trigger one slot_getsomeattrs(slot, 2) call from within qual
evaluation, and then a slot_getsomeattrs(slot, 4) from within the
projection code. If you then imagine a tuple where only the first
column exists physically, we'd copy b twice, because attno is only going
to be one 1. I think we might just want to check tts nvalid in
getmissingattrs?OK. so add something like this to the top of slot_getmissingattrs()?
startAttNum = Max(startAttNum, slot->tts_nvalid);
Yea, I think that should do the trick.
Hm, or if you want to microoptimize ;):
if (startAttNum < slot->tts_nvalid)
startAttNum = slot->tts_nvalid
I think that can use cmov (i.e. no visible branch), which Max() probably
can't usefully. Don't think the compiler can figure out that
slot->tts_nvalid cannot be smaller than startAttNum.
Greetings,
Andres Freund
On Fri, Mar 30, 2018 at 10:08 AM, Andrew Dunstan
<andrew.dunstan@2ndquadrant.com> wrote:
On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund <andres@anarazel.de> wrote:
[ missing values are loaded in the TupleDesc by RelationBuildTupleDesc ]
I'm still strongly object to do doing this unconditionally for queries
that never need any of this. We're can end up with a significant number
of large tuples in memory here, and then copy that through dozens of
tupledesc copies in queries.We're just doing the same thing we do for default values.
That's really not a defense. It's hurting us there too.
I will take a look and see if it can be avoided easily.
For missing values there are three places where we would need to load
them on demand: getmissingattr(), slot_getmissingattrs() and
expand_tuple(). However, none of those functions have obvious access
to the information required to load the missing values. We could
probably do something very ugly like getting the relid from the
TupleDesc's first attribute. Maybe someone else has a better option.
But I can't see a simple and clean way to do this.
cheers
andrew
--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
[warning, reviving a thread from 2018]
"Andrew" == Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
On Wed, Feb 21, 2018 at 7:48 PM, Andres Freund <andres@anarazel.de> wrote:
Hi,
Andrew> Comments interspersed.
@@ -4059,10 +4142,6 @@ AttrDefaultFetch(Relation relation)
systable_endscan(adscan);
heap_close(adrel, AccessShareLock);
-
- if (found != ndef)
- elog(WARNING, "%d attrdef record(s) missing for rel %s",
- ndef - found, RelationGetRelationName(relation));
}Hm, it's not obvious why this is a good thing?
I didn't find an answer to this in the thread archive, and the above
change apparently did make it into the final patch.
I just got through diagnosing a SEGV crash with someone on IRC, and the
cause turned out to be exactly this - a table had (for some reason we
could not determine within the available resources) lost its pg_attrdef
record for the one column it had with a default (which was a serial
column, so none of the fast-default code is actually implicated). Any
attempt to alter the table resulted in a crash in equalTupleDesc on this
line:
if (strcmp(defval1->adbin, defval2->adbin) != 0)
due to trying to compare adbin values which were NULL pointers.
So, questions: why was the check removed in the first place?
(Why was it previously only a warning when it causes a crash further
down the line on any alteration?)
Does equalTupleDesc need to be more defensive about this, or does the
above check need to be reinstated?
(The immediate issue was fixed by "update pg_attribute set
atthasdef=false ..." for the offending attribute and then adding it back
with ALTER TABLE, which seems to have cured the crash.)
--
Andrew (irc:RhodiumToad)
Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
I just got through diagnosing a SEGV crash with someone on IRC, and the
cause turned out to be exactly this - a table had (for some reason we
could not determine within the available resources) lost its pg_attrdef
record for the one column it had with a default (which was a serial
column, so none of the fast-default code is actually implicated). Any
attempt to alter the table resulted in a crash in equalTupleDesc on this
line:
if (strcmp(defval1->adbin, defval2->adbin) != 0)
due to trying to compare adbin values which were NULL pointers.
Ouch.
Does equalTupleDesc need to be more defensive about this, or does the
above check need to be reinstated?
Looking around at the other touches of AttrDefault.adbin in the backend
(of which there are not that many), some of them are prepared for it to be
NULL and some are not. I don't immediately have a strong opinion whether
that should be an allowed state; but if it is not allowed then it's not
okay for AttrDefaultFetch to leave such a state behind. Alternatively,
if it is allowed then equalTupleDesc is in the wrong. We should choose
one definition and make all the relevant code match it.
regards, tom lane
It reminded me of this thread, but nothing ever came of it.
/messages/by-id/20200328223052.GK20103@telsasoft.com
On 4/3/21 10:09 PM, Tom Lane wrote:
Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
I just got through diagnosing a SEGV crash with someone on IRC, and the
cause turned out to be exactly this - a table had (for some reason we
could not determine within the available resources) lost its pg_attrdef
record for the one column it had with a default (which was a serial
column, so none of the fast-default code is actually implicated).
I don't recall why the check was removed. In general the fast default
code doesn't touch adbin.
I'd be curious to know how this state came about. From the description
it sounds like something removed the pg_attrdef entry without adjusting
pg_attribute, which shouldn't happen.
Any
attempt to alter the table resulted in a crash in equalTupleDesc on this
line:
if (strcmp(defval1->adbin, defval2->adbin) != 0)
due to trying to compare adbin values which were NULL pointers.Ouch.
Does equalTupleDesc need to be more defensive about this, or does the
above check need to be reinstated?Looking around at the other touches of AttrDefault.adbin in the backend
(of which there are not that many), some of them are prepared for it to be
NULL and some are not. I don't immediately have a strong opinion whether
that should be an allowed state; but if it is not allowed then it's not
okay for AttrDefaultFetch to leave such a state behind. Alternatively,
if it is allowed then equalTupleDesc is in the wrong. We should choose
one definition and make all the relevant code match it.
There's already a warning if it sets an explicit NULL value, so I'm
inclined to say your suggestion "it's not okay for AttrDefaultFetch to
leave such a state behind" is what we should go with.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On 4/4/21 9:19 AM, Justin Pryzby wrote:
It reminded me of this thread, but nothing ever came of it.
/messages/by-id/20200328223052.GK20103@telsasoft.com
I don't recall seeing this. Around that time I was busy returning from
Australia at the start of the pandemic, so my I might have missed it in
the hubbub.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/3/21 10:09 PM, Tom Lane wrote:
Looking around at the other touches of AttrDefault.adbin in the backend
(of which there are not that many), some of them are prepared for it to be
NULL and some are not. I don't immediately have a strong opinion whether
that should be an allowed state; but if it is not allowed then it's not
okay for AttrDefaultFetch to leave such a state behind. Alternatively,
if it is allowed then equalTupleDesc is in the wrong. We should choose
one definition and make all the relevant code match it.
There's already a warning if it sets an explicit NULL value, so I'm
inclined to say your suggestion "it's not okay for AttrDefaultFetch to
leave such a state behind" is what we should go with.
Yeah, I came to the same conclusion after looking around a bit more.
There are two places in tupdesc.c that defend against adbin being NULL,
which seems a bit silly when their sibling equalTupleDesc does not.
Every other touch in the backend will segfault on a NULL value,
if it doesn't Assert first :-(
Another thing that annoyed me while I was looking at this is the
potentially O(N^2) behavior in equalTupleDesc due to not wanting
to assume that the AttrDefault arrays are in the same order.
It seems far smarter to have AttrDefaultFetch sort the arrays.
So attached is a proposed patch to clean all this up.
* Don't link the AttrDefault array into the relcache entry until
it's fully built and valid.
* elog, don't just Assert, if we fail to find an expected default
value. (Perhaps there's a case for ereport instead.)
* Remove now-useless null checks in tupdesc.c.
* Sort the AttrDefault array, remove double loops in equalTupleDesc.
I made CheckConstraintFetch likewise not install its array until
it's done. I notice that it is throwing elog(ERROR) not WARNING
for the equivalent cases of not finding the right number of
entries. I wonder if we ought to back that off to a WARNING too.
elog(ERROR) during relcache entry load is kinda nasty, because
it prevents essentially *all* use of the relation. On the other
hand, failing to enforce check constraints isn't lovely either.
Anybody have opinions about that?
regards, tom lane
Attachments:
fix-adbin-inconsistencies-1.patchtext/x-diff; charset=us-ascii; name=fix-adbin-inconsistencies-1.patchDownload
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index cb76465050..d81f6b8a60 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -174,10 +174,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
cpy->defval = (AttrDefault *) palloc(cpy->num_defval * sizeof(AttrDefault));
memcpy(cpy->defval, constr->defval, cpy->num_defval * sizeof(AttrDefault));
for (i = cpy->num_defval - 1; i >= 0; i--)
- {
- if (constr->defval[i].adbin)
- cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
- }
+ cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
}
if (constr->missing)
@@ -328,10 +325,7 @@ FreeTupleDesc(TupleDesc tupdesc)
AttrDefault *attrdef = tupdesc->constr->defval;
for (i = tupdesc->constr->num_defval - 1; i >= 0; i--)
- {
- if (attrdef[i].adbin)
- pfree(attrdef[i].adbin);
- }
+ pfree(attrdef[i].adbin);
pfree(attrdef);
}
if (tupdesc->constr->missing)
@@ -412,7 +406,6 @@ bool
equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
{
int i,
- j,
n;
if (tupdesc1->natts != tupdesc2->natts)
@@ -488,22 +481,13 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
n = constr1->num_defval;
if (n != (int) constr2->num_defval)
return false;
+ /* We assume here that both AttrDefault arrays are in adnum order */
for (i = 0; i < n; i++)
{
AttrDefault *defval1 = constr1->defval + i;
- AttrDefault *defval2 = constr2->defval;
-
- /*
- * We can't assume that the items are always read from the system
- * catalogs in the same order; so use the adnum field to identify
- * the matching item to compare.
- */
- for (j = 0; j < n; defval2++, j++)
- {
- if (defval1->adnum == defval2->adnum)
- break;
- }
- if (j >= n)
+ AttrDefault *defval2 = constr2->defval + i;
+
+ if (defval1->adnum != defval2->adnum)
return false;
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
@@ -534,25 +518,21 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
+
+ /*
+ * Similarly, we rely here on the ConstrCheck entries being sorted by
+ * name. If there are duplicate names, the outcome of the comparison
+ * is uncertain, but that should not happen.
+ */
for (i = 0; i < n; i++)
{
ConstrCheck *check1 = constr1->check + i;
- ConstrCheck *check2 = constr2->check;
-
- /*
- * Similarly, don't assume that the checks are always read in the
- * same order; match them up by name and contents. (The name
- * *should* be unique, but...)
- */
- for (j = 0; j < n; check2++, j++)
- {
- if (strcmp(check1->ccname, check2->ccname) == 0 &&
- strcmp(check1->ccbin, check2->ccbin) == 0 &&
- check1->ccvalid == check2->ccvalid &&
- check1->ccnoinherit == check2->ccnoinherit)
- break;
- }
- if (j >= n)
+ ConstrCheck *check2 = constr2->check + i;
+
+ if (!(strcmp(check1->ccname, check2->ccname) == 0 &&
+ strcmp(check1->ccbin, check2->ccbin) == 0 &&
+ check1->ccvalid == check2->ccvalid &&
+ check1->ccnoinherit == check2->ccnoinherit))
return false;
}
}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 88a68a4697..87f9bdaef0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -2501,21 +2501,24 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (attribute->atthasdef)
{
Node *this_default = NULL;
- AttrDefault *attrdef;
- int i;
/* Find default in constraint structure */
- Assert(constr != NULL);
- attrdef = constr->defval;
- for (i = 0; i < constr->num_defval; i++)
+ if (constr != NULL)
{
- if (attrdef[i].adnum == parent_attno)
+ AttrDefault *attrdef = constr->defval;
+
+ for (int i = 0; i < constr->num_defval; i++)
{
- this_default = stringToNode(attrdef[i].adbin);
- break;
+ if (attrdef[i].adnum == parent_attno)
+ {
+ this_default = stringToNode(attrdef[i].adbin);
+ break;
+ }
}
}
- Assert(this_default != NULL);
+ if (this_default == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ parent_attno, RelationGetRelationName(relation));
/*
* If it's a GENERATED default, it might contain Vars that
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b968c25dd6..2c57b200f8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1276,7 +1276,9 @@ expandTableLikeClause(RangeVar *heapRel, TableLikeClause *table_like_clause)
break;
}
}
- Assert(this_default != NULL);
+ if (this_default == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ parent_attno, RelationGetRelationName(relation));
atsubcmd = makeNode(AlterTableCmd);
atsubcmd->subtype = AT_CookedColumnDefault;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 92661abae2..531edaacdb 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1247,6 +1247,9 @@ build_column_default(Relation rel, int attrno)
break;
}
}
+ if (expr == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ attrno, RelationGetRelationName(rel));
}
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff7395c85b..b05026f1c5 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -282,7 +282,8 @@ static void RelationInitPhysicalAddr(Relation relation);
static void load_critical_index(Oid indexoid, Oid heapoid);
static TupleDesc GetPgClassDescriptor(void);
static TupleDesc GetPgIndexDescriptor(void);
-static void AttrDefaultFetch(Relation relation);
+static void AttrDefaultFetch(Relation relation, int ndef);
+static int AttrDefaultCmp(const void *a, const void *b);
static void CheckConstraintFetch(Relation relation);
static int CheckConstraintCmp(const void *a, const void *b);
static void InitIndexAmRoutine(Relation relation);
@@ -503,7 +504,6 @@ RelationBuildTupleDesc(Relation relation)
ScanKeyData skey[2];
int need;
TupleConstr *constr;
- AttrDefault *attrdef = NULL;
AttrMissing *attrmiss = NULL;
int ndef = 0;
@@ -512,8 +512,8 @@ RelationBuildTupleDesc(Relation relation)
relation->rd_rel->reltype ? relation->rd_rel->reltype : RECORDOID;
relation->rd_att->tdtypmod = -1; /* just to be sure */
- constr = (TupleConstr *) MemoryContextAlloc(CacheMemoryContext,
- sizeof(TupleConstr));
+ constr = (TupleConstr *) MemoryContextAllocZero(CacheMemoryContext,
+ sizeof(TupleConstr));
constr->has_not_null = false;
constr->has_generated_stored = false;
@@ -560,7 +560,6 @@ RelationBuildTupleDesc(Relation relation)
elog(ERROR, "invalid attribute number %d for %s",
attp->attnum, RelationGetRelationName(relation));
-
memcpy(TupleDescAttr(relation->rd_att, attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -570,22 +569,10 @@ RelationBuildTupleDesc(Relation relation)
constr->has_not_null = true;
if (attp->attgenerated == ATTRIBUTE_GENERATED_STORED)
constr->has_generated_stored = true;
-
- /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
- {
- if (attrdef == NULL)
- attrdef = (AttrDefault *)
- MemoryContextAllocZero(CacheMemoryContext,
- RelationGetNumberOfAttributes(relation) *
- sizeof(AttrDefault));
- attrdef[ndef].adnum = attnum;
- attrdef[ndef].adbin = NULL;
-
ndef++;
- }
- /* Likewise for a missing value */
+ /* If the column has a "missing" value, put it in the attrmiss array */
if (attp->atthasmissing)
{
Datum missingval;
@@ -680,33 +667,19 @@ RelationBuildTupleDesc(Relation relation)
constr->has_generated_stored ||
ndef > 0 ||
attrmiss ||
- relation->rd_rel->relchecks)
+ relation->rd_rel->relchecks > 0)
{
relation->rd_att->constr = constr;
if (ndef > 0) /* DEFAULTs */
- {
- if (ndef < RelationGetNumberOfAttributes(relation))
- constr->defval = (AttrDefault *)
- repalloc(attrdef, ndef * sizeof(AttrDefault));
- else
- constr->defval = attrdef;
- constr->num_defval = ndef;
- AttrDefaultFetch(relation);
- }
+ AttrDefaultFetch(relation, ndef);
else
constr->num_defval = 0;
constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
- {
- constr->num_check = relation->rd_rel->relchecks;
- constr->check = (ConstrCheck *)
- MemoryContextAllocZero(CacheMemoryContext,
- constr->num_check * sizeof(ConstrCheck));
CheckConstraintFetch(relation);
- }
else
constr->num_check = 0;
}
@@ -4251,21 +4224,28 @@ GetPgIndexDescriptor(void)
/*
* Load any default attribute value definitions for the relation.
+ *
+ * ndef is the number of attributes that were marked atthasdef.
+ *
+ * Note: we don't make it a hard error to be missing some pg_attrdef records.
+ * We can limp along as long as no INSERT or UPDATE needs to use the entry.
*/
static void
-AttrDefaultFetch(Relation relation)
+AttrDefaultFetch(Relation relation, int ndef)
{
- AttrDefault *attrdef = relation->rd_att->constr->defval;
- int ndef = relation->rd_att->constr->num_defval;
+ AttrDefault *attrdef;
Relation adrel;
SysScanDesc adscan;
ScanKeyData skey;
HeapTuple htup;
- Datum val;
- bool isnull;
- int found;
- int i;
+ int found = 0;
+
+ /* Allocate array with room for as many entries as expected */
+ attrdef = (AttrDefault *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ ndef * sizeof(AttrDefault));
+ /* Search pg_attrdef for relevant entries */
ScanKeyInit(&skey,
Anum_pg_attrdef_adrelid,
BTEqualStrategyNumber, F_OIDEQ,
@@ -4274,49 +4254,69 @@ AttrDefaultFetch(Relation relation)
adrel = table_open(AttrDefaultRelationId, AccessShareLock);
adscan = systable_beginscan(adrel, AttrDefaultIndexId, true,
NULL, 1, &skey);
- found = 0;
while (HeapTupleIsValid(htup = systable_getnext(adscan)))
{
Form_pg_attrdef adform = (Form_pg_attrdef) GETSTRUCT(htup);
- Form_pg_attribute attr = TupleDescAttr(relation->rd_att, adform->adnum - 1);
+ Datum val;
+ bool isnull;
- for (i = 0; i < ndef; i++)
+ /* protect limited size of array */
+ if (found >= ndef)
{
- if (adform->adnum != attrdef[i].adnum)
- continue;
- if (attrdef[i].adbin != NULL)
- elog(WARNING, "multiple attrdef records found for attr %s of rel %s",
- NameStr(attr->attname),
- RelationGetRelationName(relation));
- else
- found++;
-
- val = fastgetattr(htup,
- Anum_pg_attrdef_adbin,
- adrel->rd_att, &isnull);
- if (isnull)
- elog(WARNING, "null adbin for attr %s of rel %s",
- NameStr(attr->attname),
- RelationGetRelationName(relation));
- else
- {
- /* detoast and convert to cstring in caller's context */
- char *s = TextDatumGetCString(val);
-
- attrdef[i].adbin = MemoryContextStrdup(CacheMemoryContext, s);
- pfree(s);
- }
+ elog(WARNING, "unexpected attrdef record found for attr %d of rel %s",
+ adform->adnum, RelationGetRelationName(relation));
break;
}
- if (i >= ndef)
- elog(WARNING, "unexpected attrdef record found for attr %d of rel %s",
+ val = fastgetattr(htup,
+ Anum_pg_attrdef_adbin,
+ adrel->rd_att, &isnull);
+ if (isnull)
+ elog(WARNING, "null adbin for attr %d of rel %s",
adform->adnum, RelationGetRelationName(relation));
+ else
+ {
+ /* detoast and convert to cstring in caller's context */
+ char *s = TextDatumGetCString(val);
+
+ attrdef[found].adnum = adform->adnum;
+ attrdef[found].adbin = MemoryContextStrdup(CacheMemoryContext, s);
+ found++;
+ pfree(s);
+ }
}
systable_endscan(adscan);
table_close(adrel, AccessShareLock);
+
+ if (found != ndef)
+ elog(WARNING, "%d attrdef record(s) missing for rel %s",
+ ndef - found, RelationGetRelationName(relation));
+
+ /*
+ * Sort the AttrDefault entries by adnum, for the convenience of
+ * equalTupleDescs(). (Usually, they already will be in order, but this
+ * might not be so if systable_getnext isn't using an index.)
+ */
+ if (found > 1)
+ qsort(attrdef, found, sizeof(AttrDefault), AttrDefaultCmp);
+
+ /* Install array only after it's fully valid */
+ relation->rd_att->constr->defval = attrdef;
+ relation->rd_att->constr->num_defval = found;
+}
+
+/*
+ * qsort comparator to sort AttrDefault entries by adnum
+ */
+static int
+AttrDefaultCmp(const void *a, const void *b)
+{
+ const AttrDefault *ada = (const AttrDefault *) a;
+ const AttrDefault *adb = (const AttrDefault *) b;
+
+ return ada->adnum - adb->adnum;
}
/*
@@ -4325,14 +4325,20 @@ AttrDefaultFetch(Relation relation)
static void
CheckConstraintFetch(Relation relation)
{
- ConstrCheck *check = relation->rd_att->constr->check;
- int ncheck = relation->rd_att->constr->num_check;
+ ConstrCheck *check;
+ int ncheck = relation->rd_rel->relchecks;
Relation conrel;
SysScanDesc conscan;
ScanKeyData skey[1];
HeapTuple htup;
int found = 0;
+ /* Allocate array with room for as many entries as expected */
+ check = (ConstrCheck *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ ncheck * sizeof(ConstrCheck));
+
+ /* Search pg_constraint for relevant entries */
ScanKeyInit(&skey[0],
Anum_pg_constraint_conrelid,
BTEqualStrategyNumber, F_OIDEQ,
@@ -4353,6 +4359,7 @@ CheckConstraintFetch(Relation relation)
if (conform->contype != CONSTRAINT_CHECK)
continue;
+ /* protect limited size of array */
if (found >= ncheck)
elog(ERROR, "unexpected constraint record found for rel %s",
RelationGetRelationName(relation));
@@ -4385,9 +4392,16 @@ CheckConstraintFetch(Relation relation)
elog(ERROR, "%d constraint record(s) missing for rel %s",
ncheck - found, RelationGetRelationName(relation));
- /* Sort the records so that CHECKs are applied in a deterministic order */
- if (ncheck > 1)
- qsort(check, ncheck, sizeof(ConstrCheck), CheckConstraintCmp);
+ /*
+ * Sort the records by name. This ensures that CHECKs are applied in a
+ * deterministic order, and it also makes equalTupleDescs() faster.
+ */
+ if (found > 1)
+ qsort(check, found, sizeof(ConstrCheck), CheckConstraintCmp);
+
+ /* Install array only after it's fully valid */
+ relation->rd_att->constr->check = check;
+ relation->rd_att->constr->num_check = found;
}
/*
On 4/4/21 12:05 PM, Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/3/21 10:09 PM, Tom Lane wrote:
Looking around at the other touches of AttrDefault.adbin in the backend
(of which there are not that many), some of them are prepared for it to be
NULL and some are not. I don't immediately have a strong opinion whether
that should be an allowed state; but if it is not allowed then it's not
okay for AttrDefaultFetch to leave such a state behind. Alternatively,
if it is allowed then equalTupleDesc is in the wrong. We should choose
one definition and make all the relevant code match it.There's already a warning if it sets an explicit NULL value, so I'm
inclined to say your suggestion "it's not okay for AttrDefaultFetch to
leave such a state behind" is what we should go with.Yeah, I came to the same conclusion after looking around a bit more.
There are two places in tupdesc.c that defend against adbin being NULL,
which seems a bit silly when their sibling equalTupleDesc does not.
Every other touch in the backend will segfault on a NULL value,
if it doesn't Assert first :-(Another thing that annoyed me while I was looking at this is the
potentially O(N^2) behavior in equalTupleDesc due to not wanting
to assume that the AttrDefault arrays are in the same order.
It seems far smarter to have AttrDefaultFetch sort the arrays.
+1 The O(N^2) loops also bothered me.
So attached is a proposed patch to clean all this up.
* Don't link the AttrDefault array into the relcache entry until
it's fully built and valid.* elog, don't just Assert, if we fail to find an expected default
value. (Perhaps there's a case for ereport instead.)* Remove now-useless null checks in tupdesc.c.
* Sort the AttrDefault array, remove double loops in equalTupleDesc.
This all looks a lot cleaner and simpler to follow. I like not
allocating the array until AttrDefaultFetch.
I made CheckConstraintFetch likewise not install its array until
it's done. I notice that it is throwing elog(ERROR) not WARNING
for the equivalent cases of not finding the right number of
entries. I wonder if we ought to back that off to a WARNING too.
elog(ERROR) during relcache entry load is kinda nasty, because
it prevents essentially *all* use of the relation. On the other
hand, failing to enforce check constraints isn't lovely either.
Anybody have opinions about that?
None of this is supposed to happen, is it? If you can't fetch the
constraints (and I think of a default as almost a kind of constraint)
your database is already somehow corrupted. So maybe if any of these
things happen we should ERROR out. Anything else seems likely to corrupt
the database further.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Hi,
Thanks for the cleanup.
if (found != ncheck)
elog(ERROR, "%d constraint record(s) missing for rel %s",
ncheck - found, RelationGetRelationName(relation));
Since there is check on found being smaller than ncheck inside the loop,
the if condition can be written as:
if (found < ncheck)
Actually the above is implied by the expression, ncheck - found.
One minor comment w.r.t. the variable name is that, found is normally a
bool variable.
Maybe rename the variable nfound which would be clearer in its meaning.
+ if (found != ndef)
+ elog(WARNING, "%d attrdef record(s) missing for rel %s",
+ ndef - found, RelationGetRelationName(relation));
Since only warning is logged, there seems to be some wasted space in
attrdef. Would such space accumulate, resulting in some memory leak ?
I think the attrdef should be copied to another array of the proper size so
that there is no chance of memory leak.
Thanks
On Sun, Apr 4, 2021 at 9:05 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Show quoted text
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/3/21 10:09 PM, Tom Lane wrote:
Looking around at the other touches of AttrDefault.adbin in the backend
(of which there are not that many), some of them are prepared for it tobe
NULL and some are not. I don't immediately have a strong opinion
whether
that should be an allowed state; but if it is not allowed then it's not
okay for AttrDefaultFetch to leave such a state behind. Alternatively,
if it is allowed then equalTupleDesc is in the wrong. We should choose
one definition and make all the relevant code match it.There's already a warning if it sets an explicit NULL value, so I'm
inclined to say your suggestion "it's not okay for AttrDefaultFetch to
leave such a state behind" is what we should go with.Yeah, I came to the same conclusion after looking around a bit more.
There are two places in tupdesc.c that defend against adbin being NULL,
which seems a bit silly when their sibling equalTupleDesc does not.
Every other touch in the backend will segfault on a NULL value,
if it doesn't Assert first :-(Another thing that annoyed me while I was looking at this is the
potentially O(N^2) behavior in equalTupleDesc due to not wanting
to assume that the AttrDefault arrays are in the same order.
It seems far smarter to have AttrDefaultFetch sort the arrays.So attached is a proposed patch to clean all this up.
* Don't link the AttrDefault array into the relcache entry until
it's fully built and valid.* elog, don't just Assert, if we fail to find an expected default
value. (Perhaps there's a case for ereport instead.)* Remove now-useless null checks in tupdesc.c.
* Sort the AttrDefault array, remove double loops in equalTupleDesc.
I made CheckConstraintFetch likewise not install its array until
it's done. I notice that it is throwing elog(ERROR) not WARNING
for the equivalent cases of not finding the right number of
entries. I wonder if we ought to back that off to a WARNING too.
elog(ERROR) during relcache entry load is kinda nasty, because
it prevents essentially *all* use of the relation. On the other
hand, failing to enforce check constraints isn't lovely either.
Anybody have opinions about that?regards, tom lane
On 4/4/21 11:21 AM, Andrew Dunstan wrote:
On 4/4/21 9:19 AM, Justin Pryzby wrote:
It reminded me of this thread, but nothing ever came of it.
/messages/by-id/20200328223052.GK20103@telsasoft.comI don't recall seeing this. Around that time I was busy returning from
Australia at the start of the pandemic, so my I might have missed it in
the hubbub.
If this is still an issue, is it possible to get a self-contained test
to reproduce it?
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
On Sun, Apr 04, 2021 at 02:18:34PM -0400, Andrew Dunstan wrote:
On 4/4/21 11:21 AM, Andrew Dunstan wrote:
On 4/4/21 9:19 AM, Justin Pryzby wrote:
It reminded me of this thread, but nothing ever came of it.
/messages/by-id/20200328223052.GK20103@telsasoft.comI don't recall seeing this. Around that time I was busy returning from
Australia at the start of the pandemic, so my I might have missed it in
the hubbub.If this is still an issue, is it possible to get a self-contained test
to reproduce it?
Could you follow through with Tim on the other thread ?
--
Justin
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/4/21 12:05 PM, Tom Lane wrote:
I made CheckConstraintFetch likewise not install its array until
it's done. I notice that it is throwing elog(ERROR) not WARNING
for the equivalent cases of not finding the right number of
entries. I wonder if we ought to back that off to a WARNING too.
elog(ERROR) during relcache entry load is kinda nasty, because
it prevents essentially *all* use of the relation. On the other
hand, failing to enforce check constraints isn't lovely either.
Anybody have opinions about that?
None of this is supposed to happen, is it? If you can't fetch the
constraints (and I think of a default as almost a kind of constraint)
your database is already somehow corrupted. So maybe if any of these
things happen we should ERROR out. Anything else seems likely to corrupt
the database further.
Meh. "pg_class.relchecks is inconsistent with the number of entries
in pg_constraint" does not seem to me like a scary enough situation
to justify a panic response. Maybe there's an argument for failing
at the point where we'd need to actually apply the CHECK constraints
(similarly to what my patch is doing for missing defaults).
But preventing the user from, say, dumping the data in the table
seems to me to be making the situation worse not better.
regards, tom lane
On 4/4/21 6:50 PM, Tom Lane wrote:
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/4/21 12:05 PM, Tom Lane wrote:
I made CheckConstraintFetch likewise not install its array until
it's done. I notice that it is throwing elog(ERROR) not WARNING
for the equivalent cases of not finding the right number of
entries. I wonder if we ought to back that off to a WARNING too.
elog(ERROR) during relcache entry load is kinda nasty, because
it prevents essentially *all* use of the relation. On the other
hand, failing to enforce check constraints isn't lovely either.
Anybody have opinions about that?None of this is supposed to happen, is it? If you can't fetch the
constraints (and I think of a default as almost a kind of constraint)
your database is already somehow corrupted. So maybe if any of these
things happen we should ERROR out. Anything else seems likely to corrupt
the database further.Meh. "pg_class.relchecks is inconsistent with the number of entries
in pg_constraint" does not seem to me like a scary enough situation
to justify a panic response. Maybe there's an argument for failing
at the point where we'd need to actually apply the CHECK constraints
(similarly to what my patch is doing for missing defaults).
But preventing the user from, say, dumping the data in the table
seems to me to be making the situation worse not better.
OK, fair argument.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Andrew Dunstan <andrew@dunslane.net> writes:
On 4/4/21 6:50 PM, Tom Lane wrote:
Meh. "pg_class.relchecks is inconsistent with the number of entries
in pg_constraint" does not seem to me like a scary enough situation
to justify a panic response. Maybe there's an argument for failing
at the point where we'd need to actually apply the CHECK constraints
(similarly to what my patch is doing for missing defaults).
But preventing the user from, say, dumping the data in the table
seems to me to be making the situation worse not better.
OK, fair argument.
Here's a v2 that applies the same principles to struct ConstrCheck
as AttrDefault, ie create only valid entries (no NULL strings), and
if there's not the right number, complain at point of use rather
than failing the relcache load.
There is a hole in this, which is that if pg_class.relchecks = 0
then we won't even look in pg_constraint, so we won't realize if
there is an inconsistency. But that was true before, and I don't
want to expend an almost-always-useless catalog search to see if
relchecks = 0 is a lie.
I also tried to bring the related message texts up to something
approaching project standards, though I didn't convert them into
translatable ereports.
regards, tom lane
Attachments:
fix-adbin-inconsistencies-2.patchtext/x-diff; charset=us-ascii; name=fix-adbin-inconsistencies-2.patchDownload
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index cb76465050..ffb275afbe 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -174,10 +174,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
cpy->defval = (AttrDefault *) palloc(cpy->num_defval * sizeof(AttrDefault));
memcpy(cpy->defval, constr->defval, cpy->num_defval * sizeof(AttrDefault));
for (i = cpy->num_defval - 1; i >= 0; i--)
- {
- if (constr->defval[i].adbin)
- cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
- }
+ cpy->defval[i].adbin = pstrdup(constr->defval[i].adbin);
}
if (constr->missing)
@@ -203,10 +200,8 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
memcpy(cpy->check, constr->check, cpy->num_check * sizeof(ConstrCheck));
for (i = cpy->num_check - 1; i >= 0; i--)
{
- if (constr->check[i].ccname)
- cpy->check[i].ccname = pstrdup(constr->check[i].ccname);
- if (constr->check[i].ccbin)
- cpy->check[i].ccbin = pstrdup(constr->check[i].ccbin);
+ cpy->check[i].ccname = pstrdup(constr->check[i].ccname);
+ cpy->check[i].ccbin = pstrdup(constr->check[i].ccbin);
cpy->check[i].ccvalid = constr->check[i].ccvalid;
cpy->check[i].ccnoinherit = constr->check[i].ccnoinherit;
}
@@ -328,10 +323,7 @@ FreeTupleDesc(TupleDesc tupdesc)
AttrDefault *attrdef = tupdesc->constr->defval;
for (i = tupdesc->constr->num_defval - 1; i >= 0; i--)
- {
- if (attrdef[i].adbin)
- pfree(attrdef[i].adbin);
- }
+ pfree(attrdef[i].adbin);
pfree(attrdef);
}
if (tupdesc->constr->missing)
@@ -352,10 +344,8 @@ FreeTupleDesc(TupleDesc tupdesc)
for (i = tupdesc->constr->num_check - 1; i >= 0; i--)
{
- if (check[i].ccname)
- pfree(check[i].ccname);
- if (check[i].ccbin)
- pfree(check[i].ccbin);
+ pfree(check[i].ccname);
+ pfree(check[i].ccbin);
}
pfree(check);
}
@@ -412,7 +402,6 @@ bool
equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
{
int i,
- j,
n;
if (tupdesc1->natts != tupdesc2->natts)
@@ -488,22 +477,13 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
n = constr1->num_defval;
if (n != (int) constr2->num_defval)
return false;
+ /* We assume here that both AttrDefault arrays are in adnum order */
for (i = 0; i < n; i++)
{
AttrDefault *defval1 = constr1->defval + i;
- AttrDefault *defval2 = constr2->defval;
-
- /*
- * We can't assume that the items are always read from the system
- * catalogs in the same order; so use the adnum field to identify
- * the matching item to compare.
- */
- for (j = 0; j < n; defval2++, j++)
- {
- if (defval1->adnum == defval2->adnum)
- break;
- }
- if (j >= n)
+ AttrDefault *defval2 = constr2->defval + i;
+
+ if (defval1->adnum != defval2->adnum)
return false;
if (strcmp(defval1->adbin, defval2->adbin) != 0)
return false;
@@ -534,25 +514,21 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
n = constr1->num_check;
if (n != (int) constr2->num_check)
return false;
+
+ /*
+ * Similarly, we rely here on the ConstrCheck entries being sorted by
+ * name. If there are duplicate names, the outcome of the comparison
+ * is uncertain, but that should not happen.
+ */
for (i = 0; i < n; i++)
{
ConstrCheck *check1 = constr1->check + i;
- ConstrCheck *check2 = constr2->check;
-
- /*
- * Similarly, don't assume that the checks are always read in the
- * same order; match them up by name and contents. (The name
- * *should* be unique, but...)
- */
- for (j = 0; j < n; check2++, j++)
- {
- if (strcmp(check1->ccname, check2->ccname) == 0 &&
- strcmp(check1->ccbin, check2->ccbin) == 0 &&
- check1->ccvalid == check2->ccvalid &&
- check1->ccnoinherit == check2->ccnoinherit)
- break;
- }
- if (j >= n)
+ ConstrCheck *check2 = constr2->check + i;
+
+ if (!(strcmp(check1->ccname, check2->ccname) == 0 &&
+ strcmp(check1->ccbin, check2->ccbin) == 0 &&
+ check1->ccvalid == check2->ccvalid &&
+ check1->ccnoinherit == check2->ccnoinherit))
return false;
}
}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 88a68a4697..87f9bdaef0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -2501,21 +2501,24 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (attribute->atthasdef)
{
Node *this_default = NULL;
- AttrDefault *attrdef;
- int i;
/* Find default in constraint structure */
- Assert(constr != NULL);
- attrdef = constr->defval;
- for (i = 0; i < constr->num_defval; i++)
+ if (constr != NULL)
{
- if (attrdef[i].adnum == parent_attno)
+ AttrDefault *attrdef = constr->defval;
+
+ for (int i = 0; i < constr->num_defval; i++)
{
- this_default = stringToNode(attrdef[i].adbin);
- break;
+ if (attrdef[i].adnum == parent_attno)
+ {
+ this_default = stringToNode(attrdef[i].adbin);
+ break;
+ }
}
}
- Assert(this_default != NULL);
+ if (this_default == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ parent_attno, RelationGetRelationName(relation));
/*
* If it's a GENERATED default, it might contain Vars that
diff --git a/src/backend/executor/execMain.c b/src/backend/executor/execMain.c
index 163242f54e..9f86910a6b 100644
--- a/src/backend/executor/execMain.c
+++ b/src/backend/executor/execMain.c
@@ -1609,6 +1609,15 @@ ExecRelCheck(ResultRelInfo *resultRelInfo,
MemoryContext oldContext;
int i;
+ /*
+ * CheckConstraintFetch let this pass with only a warning, but now we
+ * should fail rather than possibly failing to enforce an important
+ * constraint.
+ */
+ if (ncheck != rel->rd_rel->relchecks)
+ elog(ERROR, "%d pg_constraint record(s) missing for relation \"%s\"",
+ rel->rd_rel->relchecks - ncheck, RelationGetRelationName(rel));
+
/*
* If first time through for this result relation, build expression
* nodetrees for rel's constraint expressions. Keep them in the per-query
@@ -1862,7 +1871,7 @@ ExecConstraints(ResultRelInfo *resultRelInfo,
}
}
- if (constr->num_check > 0)
+ if (rel->rd_rel->relchecks > 0)
{
const char *failed;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b968c25dd6..9dd30370da 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1239,8 +1239,6 @@ expandTableLikeClause(RangeVar *heapRel, TableLikeClause *table_like_clause)
(CREATE_TABLE_LIKE_DEFAULTS | CREATE_TABLE_LIKE_GENERATED)) &&
constr != NULL)
{
- AttrDefault *attrdef = constr->defval;
-
for (parent_attno = 1; parent_attno <= tupleDesc->natts;
parent_attno++)
{
@@ -1264,6 +1262,7 @@ expandTableLikeClause(RangeVar *heapRel, TableLikeClause *table_like_clause)
(table_like_clause->options & CREATE_TABLE_LIKE_DEFAULTS)))
{
Node *this_default = NULL;
+ AttrDefault *attrdef = constr->defval;
AlterTableCmd *atsubcmd;
bool found_whole_row;
@@ -1276,7 +1275,9 @@ expandTableLikeClause(RangeVar *heapRel, TableLikeClause *table_like_clause)
break;
}
}
- Assert(this_default != NULL);
+ if (this_default == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ parent_attno, RelationGetRelationName(relation));
atsubcmd = makeNode(AlterTableCmd);
atsubcmd->subtype = AT_CookedColumnDefault;
diff --git a/src/backend/rewrite/rewriteHandler.c b/src/backend/rewrite/rewriteHandler.c
index 92661abae2..da78f02775 100644
--- a/src/backend/rewrite/rewriteHandler.c
+++ b/src/backend/rewrite/rewriteHandler.c
@@ -1228,25 +1228,28 @@ build_column_default(Relation rel, int attrno)
}
/*
- * Scan to see if relation has a default for this column.
+ * If relation has a default for this column, fetch that expression.
*/
- if (att_tup->atthasdef && rd_att->constr &&
- rd_att->constr->num_defval > 0)
+ if (att_tup->atthasdef)
{
- AttrDefault *defval = rd_att->constr->defval;
- int ndef = rd_att->constr->num_defval;
-
- while (--ndef >= 0)
+ if (rd_att->constr && rd_att->constr->num_defval > 0)
{
- if (attrno == defval[ndef].adnum)
+ AttrDefault *defval = rd_att->constr->defval;
+ int ndef = rd_att->constr->num_defval;
+
+ while (--ndef >= 0)
{
- /*
- * Found it, convert string representation to node tree.
- */
- expr = stringToNode(defval[ndef].adbin);
- break;
+ if (attrno == defval[ndef].adnum)
+ {
+ /* Found it, convert string representation to node tree. */
+ expr = stringToNode(defval[ndef].adbin);
+ break;
+ }
}
}
+ if (expr == NULL)
+ elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
+ attrno, RelationGetRelationName(rel));
}
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff7395c85b..29702d6eab 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -282,7 +282,8 @@ static void RelationInitPhysicalAddr(Relation relation);
static void load_critical_index(Oid indexoid, Oid heapoid);
static TupleDesc GetPgClassDescriptor(void);
static TupleDesc GetPgIndexDescriptor(void);
-static void AttrDefaultFetch(Relation relation);
+static void AttrDefaultFetch(Relation relation, int ndef);
+static int AttrDefaultCmp(const void *a, const void *b);
static void CheckConstraintFetch(Relation relation);
static int CheckConstraintCmp(const void *a, const void *b);
static void InitIndexAmRoutine(Relation relation);
@@ -503,7 +504,6 @@ RelationBuildTupleDesc(Relation relation)
ScanKeyData skey[2];
int need;
TupleConstr *constr;
- AttrDefault *attrdef = NULL;
AttrMissing *attrmiss = NULL;
int ndef = 0;
@@ -512,8 +512,8 @@ RelationBuildTupleDesc(Relation relation)
relation->rd_rel->reltype ? relation->rd_rel->reltype : RECORDOID;
relation->rd_att->tdtypmod = -1; /* just to be sure */
- constr = (TupleConstr *) MemoryContextAlloc(CacheMemoryContext,
- sizeof(TupleConstr));
+ constr = (TupleConstr *) MemoryContextAllocZero(CacheMemoryContext,
+ sizeof(TupleConstr));
constr->has_not_null = false;
constr->has_generated_stored = false;
@@ -557,10 +557,9 @@ RelationBuildTupleDesc(Relation relation)
attnum = attp->attnum;
if (attnum <= 0 || attnum > RelationGetNumberOfAttributes(relation))
- elog(ERROR, "invalid attribute number %d for %s",
+ elog(ERROR, "invalid attribute number %d for relation \"%s\"",
attp->attnum, RelationGetRelationName(relation));
-
memcpy(TupleDescAttr(relation->rd_att, attnum - 1),
attp,
ATTRIBUTE_FIXED_PART_SIZE);
@@ -570,22 +569,10 @@ RelationBuildTupleDesc(Relation relation)
constr->has_not_null = true;
if (attp->attgenerated == ATTRIBUTE_GENERATED_STORED)
constr->has_generated_stored = true;
-
- /* If the column has a default, fill it into the attrdef array */
if (attp->atthasdef)
- {
- if (attrdef == NULL)
- attrdef = (AttrDefault *)
- MemoryContextAllocZero(CacheMemoryContext,
- RelationGetNumberOfAttributes(relation) *
- sizeof(AttrDefault));
- attrdef[ndef].adnum = attnum;
- attrdef[ndef].adbin = NULL;
-
ndef++;
- }
- /* Likewise for a missing value */
+ /* If the column has a "missing" value, put it in the attrmiss array */
if (attp->atthasmissing)
{
Datum missingval;
@@ -648,7 +635,7 @@ RelationBuildTupleDesc(Relation relation)
table_close(pg_attribute_desc, AccessShareLock);
if (need != 0)
- elog(ERROR, "catalog is missing %d attribute(s) for relid %u",
+ elog(ERROR, "pg_attribute catalog is missing %d attribute(s) for relation OID %u",
need, RelationGetRelid(relation));
/*
@@ -680,33 +667,19 @@ RelationBuildTupleDesc(Relation relation)
constr->has_generated_stored ||
ndef > 0 ||
attrmiss ||
- relation->rd_rel->relchecks)
+ relation->rd_rel->relchecks > 0)
{
relation->rd_att->constr = constr;
if (ndef > 0) /* DEFAULTs */
- {
- if (ndef < RelationGetNumberOfAttributes(relation))
- constr->defval = (AttrDefault *)
- repalloc(attrdef, ndef * sizeof(AttrDefault));
- else
- constr->defval = attrdef;
- constr->num_defval = ndef;
- AttrDefaultFetch(relation);
- }
+ AttrDefaultFetch(relation, ndef);
else
constr->num_defval = 0;
constr->missing = attrmiss;
if (relation->rd_rel->relchecks > 0) /* CHECKs */
- {
- constr->num_check = relation->rd_rel->relchecks;
- constr->check = (ConstrCheck *)
- MemoryContextAllocZero(CacheMemoryContext,
- constr->num_check * sizeof(ConstrCheck));
CheckConstraintFetch(relation);
- }
else
constr->num_check = 0;
}
@@ -4251,21 +4224,29 @@ GetPgIndexDescriptor(void)
/*
* Load any default attribute value definitions for the relation.
+ *
+ * ndef is the number of attributes that were marked atthasdef.
+ *
+ * Note: we don't make it a hard error to be missing some pg_attrdef records.
+ * We can limp along as long as nothing needs to use the default value. Code
+ * that fails to find an expected AttrDefault record should throw an error.
*/
static void
-AttrDefaultFetch(Relation relation)
+AttrDefaultFetch(Relation relation, int ndef)
{
- AttrDefault *attrdef = relation->rd_att->constr->defval;
- int ndef = relation->rd_att->constr->num_defval;
+ AttrDefault *attrdef;
Relation adrel;
SysScanDesc adscan;
ScanKeyData skey;
HeapTuple htup;
- Datum val;
- bool isnull;
- int found;
- int i;
+ int found = 0;
+
+ /* Allocate array with room for as many entries as expected */
+ attrdef = (AttrDefault *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ ndef * sizeof(AttrDefault));
+ /* Search pg_attrdef for relevant entries */
ScanKeyInit(&skey,
Anum_pg_attrdef_adrelid,
BTEqualStrategyNumber, F_OIDEQ,
@@ -4274,65 +4255,94 @@ AttrDefaultFetch(Relation relation)
adrel = table_open(AttrDefaultRelationId, AccessShareLock);
adscan = systable_beginscan(adrel, AttrDefaultIndexId, true,
NULL, 1, &skey);
- found = 0;
while (HeapTupleIsValid(htup = systable_getnext(adscan)))
{
Form_pg_attrdef adform = (Form_pg_attrdef) GETSTRUCT(htup);
- Form_pg_attribute attr = TupleDescAttr(relation->rd_att, adform->adnum - 1);
+ Datum val;
+ bool isnull;
- for (i = 0; i < ndef; i++)
+ /* protect limited size of array */
+ if (found >= ndef)
{
- if (adform->adnum != attrdef[i].adnum)
- continue;
- if (attrdef[i].adbin != NULL)
- elog(WARNING, "multiple attrdef records found for attr %s of rel %s",
- NameStr(attr->attname),
- RelationGetRelationName(relation));
- else
- found++;
-
- val = fastgetattr(htup,
- Anum_pg_attrdef_adbin,
- adrel->rd_att, &isnull);
- if (isnull)
- elog(WARNING, "null adbin for attr %s of rel %s",
- NameStr(attr->attname),
- RelationGetRelationName(relation));
- else
- {
- /* detoast and convert to cstring in caller's context */
- char *s = TextDatumGetCString(val);
-
- attrdef[i].adbin = MemoryContextStrdup(CacheMemoryContext, s);
- pfree(s);
- }
+ elog(WARNING, "unexpected pg_attrdef record found for attribute %d of relation \"%s\"",
+ adform->adnum, RelationGetRelationName(relation));
break;
}
- if (i >= ndef)
- elog(WARNING, "unexpected attrdef record found for attr %d of rel %s",
+ val = fastgetattr(htup,
+ Anum_pg_attrdef_adbin,
+ adrel->rd_att, &isnull);
+ if (isnull)
+ elog(WARNING, "null adbin for attribute %d of relation \"%s\"",
adform->adnum, RelationGetRelationName(relation));
+ else
+ {
+ /* detoast and convert to cstring in caller's context */
+ char *s = TextDatumGetCString(val);
+
+ attrdef[found].adnum = adform->adnum;
+ attrdef[found].adbin = MemoryContextStrdup(CacheMemoryContext, s);
+ pfree(s);
+ found++;
+ }
}
systable_endscan(adscan);
table_close(adrel, AccessShareLock);
+
+ if (found != ndef)
+ elog(WARNING, "%d pg_attrdef record(s) missing for relation \"%s\"",
+ ndef - found, RelationGetRelationName(relation));
+
+ /*
+ * Sort the AttrDefault entries by adnum, for the convenience of
+ * equalTupleDescs(). (Usually, they already will be in order, but this
+ * might not be so if systable_getnext isn't using an index.)
+ */
+ if (found > 1)
+ qsort(attrdef, found, sizeof(AttrDefault), AttrDefaultCmp);
+
+ /* Install array only after it's fully valid */
+ relation->rd_att->constr->defval = attrdef;
+ relation->rd_att->constr->num_defval = found;
+}
+
+/*
+ * qsort comparator to sort AttrDefault entries by adnum
+ */
+static int
+AttrDefaultCmp(const void *a, const void *b)
+{
+ const AttrDefault *ada = (const AttrDefault *) a;
+ const AttrDefault *adb = (const AttrDefault *) b;
+
+ return ada->adnum - adb->adnum;
}
/*
* Load any check constraints for the relation.
+ *
+ * As with defaults, if we don't find the expected number of them, just warn
+ * here. The executor should throw an error if an INSERT/UPDATE is attempted.
*/
static void
CheckConstraintFetch(Relation relation)
{
- ConstrCheck *check = relation->rd_att->constr->check;
- int ncheck = relation->rd_att->constr->num_check;
+ ConstrCheck *check;
+ int ncheck = relation->rd_rel->relchecks;
Relation conrel;
SysScanDesc conscan;
ScanKeyData skey[1];
HeapTuple htup;
int found = 0;
+ /* Allocate array with room for as many entries as expected */
+ check = (ConstrCheck *)
+ MemoryContextAllocZero(CacheMemoryContext,
+ ncheck * sizeof(ConstrCheck));
+
+ /* Search pg_constraint for relevant entries */
ScanKeyInit(&skey[0],
Anum_pg_constraint_conrelid,
BTEqualStrategyNumber, F_OIDEQ,
@@ -4347,15 +4357,18 @@ CheckConstraintFetch(Relation relation)
Form_pg_constraint conform = (Form_pg_constraint) GETSTRUCT(htup);
Datum val;
bool isnull;
- char *s;
/* We want check constraints only */
if (conform->contype != CONSTRAINT_CHECK)
continue;
+ /* protect limited size of array */
if (found >= ncheck)
- elog(ERROR, "unexpected constraint record found for rel %s",
+ {
+ elog(WARNING, "unexpected pg_constraint record found for relation \"%s\"",
RelationGetRelationName(relation));
+ break;
+ }
check[found].ccvalid = conform->convalidated;
check[found].ccnoinherit = conform->connoinherit;
@@ -4367,27 +4380,36 @@ CheckConstraintFetch(Relation relation)
Anum_pg_constraint_conbin,
conrel->rd_att, &isnull);
if (isnull)
- elog(ERROR, "null conbin for rel %s",
+ elog(WARNING, "null conbin for relation \"%s\"",
RelationGetRelationName(relation));
+ else
+ {
+ /* detoast and convert to cstring in caller's context */
+ char *s = TextDatumGetCString(val);
- /* detoast and convert to cstring in caller's context */
- s = TextDatumGetCString(val);
- check[found].ccbin = MemoryContextStrdup(CacheMemoryContext, s);
- pfree(s);
-
- found++;
+ check[found].ccbin = MemoryContextStrdup(CacheMemoryContext, s);
+ pfree(s);
+ found++;
+ }
}
systable_endscan(conscan);
table_close(conrel, AccessShareLock);
if (found != ncheck)
- elog(ERROR, "%d constraint record(s) missing for rel %s",
+ elog(WARNING, "%d pg_constraint record(s) missing for relation \"%s\"",
ncheck - found, RelationGetRelationName(relation));
- /* Sort the records so that CHECKs are applied in a deterministic order */
- if (ncheck > 1)
- qsort(check, ncheck, sizeof(ConstrCheck), CheckConstraintCmp);
+ /*
+ * Sort the records by name. This ensures that CHECKs are applied in a
+ * deterministic order, and it also makes equalTupleDescs() faster.
+ */
+ if (found > 1)
+ qsort(check, found, sizeof(ConstrCheck), CheckConstraintCmp);
+
+ /* Install array only after it's fully valid */
+ relation->rd_att->constr->check = check;
+ relation->rd_att->constr->num_check = found;
}
/*
Zhihong Yu <zyu@yugabyte.com> writes:
if (found != ncheck)
Since there is check on found being smaller than ncheck inside the loop,
the if condition can be written as:
if (found < ncheck)
Doesn't really seem like an improvement? Nor am I excited about renaming
these "found" variables, considering that those names can be traced back
more than twenty years.
+ if (found != ndef) + elog(WARNING, "%d attrdef record(s) missing for rel %s", + ndef - found, RelationGetRelationName(relation));
Since only warning is logged, there seems to be some wasted space in
attrdef. Would such space accumulate, resulting in some memory leak ?
No, it's just one palloc chunk either way. I do not think there is
any value in adding more code to reclaim the unused array slots a
bit sooner. (Because of aset.c's power-of-two allocation practices,
it's likely there would be zero actual space savings anyway.)
regards, tom lane
"Andrew" == Andrew Dunstan <andrew@dunslane.net> writes:
Andrew> I'd be curious to know how this state came about.
Me too, but available information is fairly sparse: PG 12.5, in a
container, backing a (personal) instance of Gitlab; the only database
admin operations should have been those done by Gitlab itself, but I
have not audited that codebase. No information on any history of
crashes. The missing pg_attrdef row appeared to be missing or not
visible in the heap, not just missing from indexes; it did not show up
in queries whether seqscan or indexscan was used. Available time did not
permit trying to use pageinspect on pg_attrdef.
This gitlab ticket refers to the same incident:
https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6047
(which actually contains a new relevant fact that hadn't been mentioned
in the IRC discussion, which is that the problem affected multiple
tables, not just one.)
--
Andrew (irc:RhodiumToad)
Andrew Gierth <andrew@tao11.riddles.org.uk> writes:
This gitlab ticket refers to the same incident:
https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6047
(which actually contains a new relevant fact that hadn't been mentioned
in the IRC discussion, which is that the problem affected multiple
tables, not just one.)
Hmm. If you assume that the problem is either a corrupted index on
pg_attrdef, or some block(s) of pg_attrdef got zeroed out, then it
would be entirely unsurprising for multiple tables to be affected.
I've pushed the v2 patch to HEAD. Not sure if we want to consider
back-patching.
regards, tom lane