support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

Started by jian heabout 1 year ago13 messageshackers
Jump to latest
#1jian he
jian.universality@gmail.com

hi.

in RememberAllDependentForRebuilding
while (HeapTupleIsValid(depTup = systable_getnext(scan)))
{
if(subtype == AT_SetExpression)
elog(INFO, "foundObject.classId:%d", foundObject.classId);
}
Then do the regress test on generated_stored.sql
I found out only constraints and indexes will be rebuilt
while we are doing ALTER TABLE ALTER COLUMN SET EXPRESSION.

we can also see RememberAllDependentForRebuilding handling of:
case RelationRelationId:
case AttrDefaultRelationId:
case ConstraintRelationId:

RememberAllDependentForRebuilding record
AlteredTableInfo->changedConstraintOids, AlteredTableInfo->changedIndexOids.
ATPostAlterTypeCleanup will construct steps to rebuild these
constraints over the generated column.
and if these constraints are successfully installed,
AlteredTableInfo->constraints will be populated.
then in phase3 ATRewriteTable will do the scan or rewrite.

in summary: ATExecSetExpression, RememberAllDependentForRebuilding
will do all the work to change the generation expression,
whether it's virtual or stored.

we didn't support virtual generated columns over domain with check constraint.
we also didn't support index over virtual generated columns.
to support change generation expressions for virtual generated columns
over check constraints,
the code seems not hard.

Attachments:

v1-0001-virtual-generated-column-set-expression-with-check-constr.patchtext/x-patch; charset=US-ASCII; name=v1-0001-virtual-generated-column-set-expression-with-check-constr.patchDownload+58-28
#2jian he
jian.universality@gmail.com
In reply to: jian he (#1)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Tue, Mar 11, 2025 at 12:17 PM jian he <jian.universality@gmail.com> wrote:

hi.
in summary: ATExecSetExpression, RememberAllDependentForRebuilding
will do all the work to change the generation expression,
whether it's virtual or stored.

while working on another patch, I found out this can be further simplified.
Thus a new patch is attached.

Attachments:

v1-0001-generated-column-set-expression-with-check-constraint.patchtext/x-patch; charset=US-ASCII; name=v1-0001-generated-column-set-expression-with-check-constraint.patchDownload+51-31
#3jian he
jian.universality@gmail.com
In reply to: jian he (#2)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

hi.
rebased only.

Attachments:

v2-0001-allow-change-generated-expression-while-check-constraint-.patchtext/x-patch; charset=US-ASCII; name=v2-0001-allow-change-generated-expression-while-check-constraint-.patchDownload+48-28
#4Matheus Alcantara
matheusssilv97@gmail.com
In reply to: jian he (#3)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

Hi,

On 06/07/25 23:24, jian he wrote:

hi.
rebased only.

+--test no table rewrite happen
+ALTER TABLE gtest20 ALTER COLUMN b SET EXPRESSION AS (a * 4), ADD 
COLUMN C int default 11;
+SELECT  pa.attnum,pa.attname,attmissingval
+FROM    pg_attribute pa
+JOIN    pg_attrdef patt ON pa.attrelid = patt.adrelid AND pa.attnum = 
patt.adnum
+WHERE   pa.attrelid = 'gtest20'::regclass

Not sure if I understand the goal of this query. Is to check if the
table was not rewritten after an update on the virtual generated
column? If that's the case, I've tested this query by updating the
expression from a STORED generated column and it return the same
results, before and after the alter table, so I'm not sure if it's the
best way to test this, or I'm missing something?

Perhaps we could use pg_relation_filenode() and use \gset to store the
value on a variable before the ALTER TABLE execution and check if the
value is the same after the ALTER TABLE SET EXPRESSION is executed.
Please see the attached diff for reference.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

Attachments:

test-no-table-rewrite.diff.nocfbottext/plain; charset=UTF-8; name=test-no-table-rewrite.diff.nocfbotDownload+8-14
#5jian he
jian.universality@gmail.com
In reply to: Matheus Alcantara (#4)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Thu, Jan 15, 2026 at 5:02 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

Please see the attached diff for reference.

hi.
Your patch made the test more simple.
so i added a ``\d gtest20``

I aslo polished the commit message.

--
jian
https://www.enterprisedb.com/

Attachments:

v3-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchtext/x-patch; charset=US-ASCII; name=v3-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchDownload+54-28
#6Matheus Alcantara
matheusssilv97@gmail.com
In reply to: jian he (#5)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Thu Jan 15, 2026 at 1:31 AM -03, jian he wrote:

On Thu, Jan 15, 2026 at 5:02 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

Please see the attached diff for reference.

hi.
Your patch made the test more simple.
so i added a ``\d gtest20``

I aslo polished the commit message.

Thanks for the new version. The commit message seems better. Just a few
comments:

+	/*
+	 * Find everything that depends on the column (constraints, indexes, etc),
+	 * and record enough information to let us recreate the objects after
+	 * rewrite.
+	 */
+	RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName);
+
Perhaps this comments should be updated since we are now collecting
these dependencies for virtual generated columns too that it not require
a table rewrite.

---

I think that it would be good to update the SET EXPRESSION AS
documentation on doc/src/sgml/ref/alter_table.sgml to mention that for
virtual columns the table is not rewritten but a full table scan may
still be needed if the column has check constraints.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

#7jian he
jian.universality@gmail.com
In reply to: Matheus Alcantara (#6)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Thu, Jan 15, 2026 at 11:29 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

+       /*
+        * Find everything that depends on the column (constraints, indexes, etc),
+        * and record enough information to let us recreate the objects after
+        * rewrite.
+        */
+       RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName);
+
Perhaps this comments should be updated since we are now collecting
these dependencies for virtual generated columns too that it not require
a table rewrite.

---

I guess we can change it as:

/*
* Find everything that depends on the column (constraints, indexes, etc),
* and record enough information to let us recreate the objects.
*/
RememberAllDependentForRebuilding(tab, AT_SetExpression, rel,
attnum, colName);

I think that it would be good to update the SET EXPRESSION AS
documentation on doc/src/sgml/ref/alter_table.sgml to mention that for
virtual columns the table is not rewritten but a full table scan may
still be needed if the column has check constraints.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

In doc/src/sgml/ref/alter_table.sgml, I intended to change it as follows:

   <varlistentry id="sql-altertable-desc-set-expression">
    <term><literal>SET EXPRESSION AS</literal></term>
    <listitem>
     <para>
      This form replaces the expression of a generated column.  Existing data
      in a stored generated column is rewritten and all the future changes
      will apply the new generation expression.
+      Virtual generated columns do not require a table rewrite.
+      However if the column is part of a <literal>CHECK</literal> constraint
+      expression, the constraint will be rebuilt, requiring a table scan to
+      ensure that existing rows meet the constraint.
     </para>
#8Matheus Alcantara
matheusssilv97@gmail.com
In reply to: jian he (#7)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On 16/01/26 08:31, jian he wrote:

On Thu, Jan 15, 2026 at 11:29 PM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

+       /*
+        * Find everything that depends on the column (constraints, indexes, etc),
+        * and record enough information to let us recreate the objects after
+        * rewrite.
+        */
+       RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName);
+
Perhaps this comments should be updated since we are now collecting
these dependencies for virtual generated columns too that it not require
a table rewrite.

---

I guess we can change it as:

/*
* Find everything that depends on the column (constraints, indexes, etc),
* and record enough information to let us recreate the objects.
*/
RememberAllDependentForRebuilding(tab, AT_SetExpression, rel,
attnum, colName);

It looks good.

I think that it would be good to update the SET EXPRESSION AS
documentation on doc/src/sgml/ref/alter_table.sgml to mention that for
virtual columns the table is not rewritten but a full table scan may
still be needed if the column has check constraints.

In doc/src/sgml/ref/alter_table.sgml, I intended to change it as follows:

<varlistentry id="sql-altertable-desc-set-expression">
<term><literal>SET EXPRESSION AS</literal></term>
<listitem>
<para>
This form replaces the expression of a generated column.  Existing data
in a stored generated column is rewritten and all the future changes
will apply the new generation expression.
+      Virtual generated columns do not require a table rewrite.
+      However if the column is part of a <literal>CHECK</literal> constraint
+      expression, the constraint will be rebuilt, requiring a table scan to
+      ensure that existing rows meet the constraint.
</para>

Sounds good.

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

#9jian he
jian.universality@gmail.com
In reply to: Matheus Alcantara (#8)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Sat, Jan 17, 2026 at 12:07 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

<varlistentry id="sql-altertable-desc-set-expression">
<term><literal>SET EXPRESSION AS</literal></term>
<listitem>
<para>
This form replaces the expression of a generated column.  Existing data
in a stored generated column is rewritten and all the future changes
will apply the new generation expression.
+      Virtual generated columns do not require a table rewrite.
+      However if the column is part of a <literal>CHECK</literal> constraint
+      expression, the constraint will be rebuilt, requiring a table scan to
+      ensure that existing rows meet the constraint.
</para>

Sounds good.

hi.

I ultimately settled on the following:
+      Replacing the expression of a virtual generated columns do not require a
+      table rewrite, however if the column is part of a
<literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, requiring a table
+      scan to ensure that existing rows meet the constraint.

--
jian
https://www.enterprisedb.com

Attachments:

v4-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchDownload+57-28
#10Matheus Alcantara
matheusssilv97@gmail.com
In reply to: jian he (#9)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Mon Jan 19, 2026 at 1:11 PM -03, jian he wrote:

I ultimately settled on the following:
+      Replacing the expression of a virtual generated columns do not require a
+      table rewrite, however if the column is part of a
<literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, requiring a table
+      scan to ensure that existing rows meet the constraint.

Thanks for the new version. Just a minor comment

+      Replacing the expression of a virtual generated columns do not require a
+      table rewrite, however if the column is part of a <literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, requiring a table
+      scan to ensure that existing rows meet the constraint.

I think that it sounds better with: "Replacing the expression of a
virtual generated column does not require a table rewrite..."

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

#11jian he
jian.universality@gmail.com
In reply to: Matheus Alcantara (#10)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On Wed, Jan 21, 2026 at 2:48 AM Matheus Alcantara
<matheusssilv97@gmail.com> wrote:

Thanks for the new version. Just a minor comment

+      Replacing the expression of a virtual generated columns do not require a
+      table rewrite, however if the column is part of a <literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, requiring a table
+      scan to ensure that existing rows meet the constraint.

I think that it sounds better with: "Replacing the expression of a
virtual generated column does not require a table rewrite..."

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

sure. please check the attached.

--
jian
https://www.enterprisedb.com

Attachments:

v5-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchtext/x-patch; charset=US-ASCII; name=v5-0001-Allow-ALTER-COLUMN-SET-EXPRESSION-on-virtual-columns-with-CHECK-c.patchDownload+57-28
#12Matheus Alcantara
matheusssilv97@gmail.com
In reply to: jian he (#11)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On 21/01/26 02:10, jian he wrote:

+      Replacing the expression of a virtual generated columns do not require a
+      table rewrite, however if the column is part of a <literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, requiring a table
+      scan to ensure that existing rows meet the constraint.

I think that it sounds better with: "Replacing the expression of a
virtual generated column does not require a table rewrite..."

sure. please check the attached.

Thanks. The patch is in a good shape for me, I don't have any other
comments. Perhaps we could move the CF status for "Ready for Committer"?

--
Matheus Alcantara
EDB: https://www.enterprisedb.com

#13Peter Eisentraut
peter_e@gmx.net
In reply to: Matheus Alcantara (#12)
Re: support ALTER COLUMN SET EXPRESSION over virtual generated column with check constraint

On 21.01.26 20:26, Matheus Alcantara wrote:

On 21/01/26 02:10, jian he wrote:

+      Replacing the expression of a virtual generated columns do not 
require a
+      table rewrite, however if the column is part of a 
<literal>CHECK</literal>
+      constraint expression, the constraint will be rebuilt, 
requiring a table
+      scan to ensure that existing rows meet the constraint.

I think that it sounds better with: "Replacing the expression of a
virtual generated column does not require a table rewrite..."

sure. please check the attached.

Thanks. The patch is in a good shape for me, I don't have any other
comments. Perhaps we could move the CF status for "Ready for Committer"?

(As a matter of process, if you, as the reviewer, feel like you have
reviewed all you could, then it is appropriate for you to change the
status to "Ready for Committer".)

I have committed this.

I changed the wording in the documentation a bit, mentioning just
"constraints" in general, since this also applies to the existing
support for not-null constraints, and it would presumably apply to other
kinds of constraints as well, if we want to support those.