REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Started by Nathan Bossartabout 2 months ago14 messageshackers
Beta feature

Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.

won't retrysuccessCI history

This thread has been committed, so CI has stopped here. Anything below is the last result it produced.

You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:

docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t253571
psql -h localhost -U postgres

Built from patchset v13 (message #13), September 09, 2026 at 05:40 PM.

Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:

git clone --branch t253571_13 https://github.com/hackorum-dev/postgres.git

In a checkout you already have, add the fork once:

git remote add hackorum https://github.com/hackorum-dev/postgres.git

then, for this patchset and every later one:

git fetch hackorum t253571_13 && git checkout t253571_13

Patchset v13 (message #13) is on t253571_13

Jump to latest
#1Nathan Bossart
nathandbossart@gmail.com

IIUC this can cause problems for logical replication consumers, since the
CONCURRENTLY path doesn't go through the rewriteheap.c machinery like the
non-concurrent path does:

CREATE TABLE t (a INT PRIMARY KEY, b TEXT) WITH (user_catalog_table = true);
ALTER TABLE t ALTER COLUMN b TYPE varchar(100); -- correctly fails
REPACK (CONCURRENTLY) t; -- should fail but doesn't

I believe the fix for v19 is to add a check for
RelationIsUsedAsCatalogTable() in check_concurrent_repack_requirements()
(plus a corresponding update to the list of CONCURRENTLY limitations in the
docs).

--
nathan

#2Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Nathan Bossart (#1)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Thu, Aug 27, 2026 at 7:22 AM Nathan Bossart <nathandbossart@gmail.com> wrote:

IIUC this can cause problems for logical replication consumers, since the
CONCURRENTLY path doesn't go through the rewriteheap.c machinery like the
non-concurrent path does:

CREATE TABLE t (a INT PRIMARY KEY, b TEXT) WITH (user_catalog_table = true);
ALTER TABLE t ALTER COLUMN b TYPE varchar(100); -- correctly fails
REPACK (CONCURRENTLY) t; -- should fail but doesn't

I believe the fix for v19 is to add a check for
RelationIsUsedAsCatalogTable() in check_concurrent_repack_requirements()
(plus a corresponding update to the list of CONCURRENTLY limitations in the
docs).

That's correct. I verified that concurrent repack doesn't write the
pg_logical/mappings files, which decoding later needs to correctly map
the old tuple locations to the new ones after a rewrite, unlike
non-concurrent CLUSTER, VACUUM FULL and REPACK on a user catalog
table. I think restricting user catalog tables, rather than making
concurrent repack support them, is the right choice at this point in
the release cycle.

Please find attached a patch with the suggested doc change. I didn't
add a test, I don't think we need one.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_2
v1-0001-Restrict-concurrent-repack-on-user-catalog-tables.patchapplication/octet-stream; name=v1-0001-Restrict-concurrent-repack-on-user-catalog-tables.patchDownload+17-1
#3Antonin Houska
ah@cybertec.at
In reply to: Bharath Rupireddy (#2)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> wrote:

Hi,

On Thu, Aug 27, 2026 at 7:22 AM Nathan Bossart <nathandbossart@gmail.com> wrote:

IIUC this can cause problems for logical replication consumers, since the
CONCURRENTLY path doesn't go through the rewriteheap.c machinery like the
non-concurrent path does:

CREATE TABLE t (a INT PRIMARY KEY, b TEXT) WITH (user_catalog_table = true);
ALTER TABLE t ALTER COLUMN b TYPE varchar(100); -- correctly fails
REPACK (CONCURRENTLY) t; -- should fail but doesn't

I believe the fix for v19 is to add a check for
RelationIsUsedAsCatalogTable() in check_concurrent_repack_requirements()
(plus a corresponding update to the list of CONCURRENTLY limitations in the
docs).

That's correct. I verified that concurrent repack doesn't write the
pg_logical/mappings files, which decoding later needs to correctly map
the old tuple locations to the new ones after a rewrite, unlike
non-concurrent CLUSTER, VACUUM FULL and REPACK on a user catalog
table. I think restricting user catalog tables, rather than making
concurrent repack support them, is the right choice at this point in
the release cycle.

Please find attached a patch with the suggested doc change. I didn't
add a test, I don't think we need one.

Thanks! I'm just thinking about this comment:

+ /* The CONCURRENTLY path does not write logical rewrite mappings. */

IMO it makes no sense to rewrite the logical mappings as long as REPACK
(CONCURRENTLY) changes visibility information (i.e. it's MVCC-unsafe). Once we
implement the MVCC-safety, we should not remove this check unless we implement
the rewriting of the mappings for user catalog tables - that might also be
worth mentioning in the comment.

So far, I'd consider the MVCC-unsafety the primary reason to prohibit REPACK
(CONCURRENTLY) (or anyother MVCC-unsafe command) on the user catalog tables,
because it can make the contents of those tables invisible to the output
plugin at some point.

--
Antonin Houska
Web: https://www.cybertec-postgresql.com

#4Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Antonin Houska (#3)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Sat, Aug 29, 2026 at 4:23 AM Antonin Houska <ah@cybertec.at> wrote:

Thanks! I'm just thinking about this comment:

+ /* The CONCURRENTLY path does not write logical rewrite mappings. */

IMO it makes no sense to rewrite the logical mappings as long as REPACK
(CONCURRENTLY) changes visibility information (i.e. it's MVCC-unsafe). Once we
implement the MVCC-safety, we should not remove this check unless we implement
the rewriting of the mappings for user catalog tables - that might also be
worth mentioning in the comment.

So far, I'd consider the MVCC-unsafety the primary reason to prohibit REPACK
(CONCURRENTLY) (or anyother MVCC-unsafe command) on the user catalog tables,
because it can make the contents of those tables invisible to the output
plugin at some point.

Thanks for pointing this out. I read the docs related to MVCC-safety
of concurrent repack. I reworded the comment as you suggested. Feel
free to suggest changes to it if any. Please find the attached v2
patch.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_4
v2-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchapplication/octet-stream; name=v2-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchDownload+23-1
#5Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Bharath Rupireddy (#4)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Sat, Aug 29, 2026 at 8:34 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

Hi,

On Sat, Aug 29, 2026 at 4:23 AM Antonin Houska <ah@cybertec.at> wrote:

Thanks! I'm just thinking about this comment:

+ /* The CONCURRENTLY path does not write logical rewrite mappings. */

IMO it makes no sense to rewrite the logical mappings as long as REPACK
(CONCURRENTLY) changes visibility information (i.e. it's MVCC-unsafe). Once we
implement the MVCC-safety, we should not remove this check unless we implement
the rewriting of the mappings for user catalog tables - that might also be
worth mentioning in the comment.

So far, I'd consider the MVCC-unsafety the primary reason to prohibit REPACK
(CONCURRENTLY) (or anyother MVCC-unsafe command) on the user catalog tables,
because it can make the contents of those tables invisible to the output
plugin at some point.

Thanks for pointing this out. I read the docs related to MVCC-safety
of concurrent repack. I reworded the comment as you suggested. Feel
free to suggest changes to it if any. Please find the attached v2
patch.

Please find the attached v3 patch with a test added similar to other
restricted error cases. Sorry for the noise.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_5
v3-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchapplication/octet-stream; name=v3-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchDownload+34-1
#6Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bharath Rupireddy (#5)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

On Mon, Aug 31, 2026 at 8:35 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

Hi,

On Sat, Aug 29, 2026 at 8:34 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

Hi,

On Sat, Aug 29, 2026 at 4:23 AM Antonin Houska <ah@cybertec.at> wrote:

Thanks! I'm just thinking about this comment:

+ /* The CONCURRENTLY path does not write logical rewrite mappings. */

IMO it makes no sense to rewrite the logical mappings as long as REPACK
(CONCURRENTLY) changes visibility information (i.e. it's MVCC-unsafe). Once we
implement the MVCC-safety, we should not remove this check unless we implement
the rewriting of the mappings for user catalog tables - that might also be
worth mentioning in the comment.

So far, I'd consider the MVCC-unsafety the primary reason to prohibit REPACK
(CONCURRENTLY) (or anyother MVCC-unsafe command) on the user catalog tables,
because it can make the contents of those tables invisible to the output
plugin at some point.

Thanks for pointing this out. I read the docs related to MVCC-safety
of concurrent repack. I reworded the comment as you suggested. Feel
free to suggest changes to it if any. Please find the attached v2
patch.

Please find the attached v3 patch with a test added similar to other
restricted error cases. Sorry for the noise.

I reviewed the patch. It's simple and I agree to add a check for user
catalog tables. I've slightly rephrased the comment in
check_concurrent_repack_requirements() to clarify the reason why
REPACK (CONCURRENTLY) doesn't support user catalog tables while it
does for normal tables even though it's a MVCC-unsafe operation. Also
updated the commit message as well. Please review it.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_6
v4-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchDownload+33-1
#7Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Masahiko Sawada (#6)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Fri, Sep 4, 2026 at 11:02 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

I reviewed the patch. It's simple and I agree to add a check for user
catalog tables. I've slightly rephrased the comment in
check_concurrent_repack_requirements() to clarify the reason why
REPACK (CONCURRENTLY) doesn't support user catalog tables while it
does for normal tables even though it's a MVCC-unsafe operation. Also
updated the commit message as well. Please review it.

Thanks for reviewing it. The v4 patch looks good to me. pgindent and
make check are happy.

A nit:

Code comment:
+ * tables under a historic snapshot. Removing this check requires making
+ * it MVCC-safe and logical rewrite mappings.
+ */

Commit message:
Disallow REPACK (CONCURRENTLY) on user catalog tables. Removing this
check requires making it MVCC-safe and logical rewrite mappings.

"Making it logical rewrite mappings" doesn't read correctly. How about
"Removing this check requires making it MVCC-safe and adding logical
rewrite mapping support" or such?

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

#8Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Masahiko Sawada (#6)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hello,

On 2026-Sep-04, Masahiko Sawada wrote:

I reviewed the patch. It's simple and I agree to add a check for user
catalog tables. I've slightly rephrased the comment in
check_concurrent_repack_requirements() to clarify the reason why
REPACK (CONCURRENTLY) doesn't support user catalog tables while it
does for normal tables even though it's a MVCC-unsafe operation. Also
updated the commit message as well. Please review it.

Thanks, I gave this a look and I think it's okay. I'm not sure about
the exact wording of the error message though,

+		ereport(ERROR,
+				errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+				errmsg("cannot execute %s on relation \"%s\"",
+					   "REPACK (CONCURRENTLY)", RelationGetRelationName(rel)),
+				errhint("%s is not supported for tables used as catalog tables.",
+						"REPACK (CONCURRENTLY)"));

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
HINT: REPACK (CONCURRENTLY) is not supported for tables used as catalog tables.

I think the HINT should really be DETAIL, and I'm not sure we need to
mention the command again there. On the other hand, I'm not really
happy with the term "tables used as catalog tables". In the
documentation we use the term "user provided catalog tables" or simply
"user catalog tables" (see <sect2 id="logicaldecoding-capabilities">).
So how about something like this?

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
DETAIL: This operation is not supported for user catalog tables.

Would you push this patch, or do you want me to?

Thanks!

--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/

#9Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Alvaro Herrera (#8)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Tue, Sep 8, 2026 at 5:47 AM Alvaro Herrera <alvherre@kurilemu.de> wrote:

Thanks, I gave this a look and I think it's okay. I'm not sure about
the exact wording of the error message though,

Thanks for reviewing it.

+             ereport(ERROR,
+                             errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                             errmsg("cannot execute %s on relation \"%s\"",
+                                        "REPACK (CONCURRENTLY)", RelationGetRelationName(rel)),
+                             errhint("%s is not supported for tables used as catalog tables.",
+                                             "REPACK (CONCURRENTLY)"));

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
HINT: REPACK (CONCURRENTLY) is not supported for tables used as catalog tables.

I think the HINT should really be DETAIL, and I'm not sure we need to
mention the command again there.

On the other hand, I'm not really
happy with the term "tables used as catalog tables". In the
documentation we use the term "user provided catalog tables" or simply
"user catalog tables" (see <sect2 id="logicaldecoding-capabilities">).
So how about something like this?

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
DETAIL: This operation is not supported for user catalog tables.

I agree that this is not a hint message. If we ever say something like
"Please do something to get repack to work on user catalog tables,"
that would be a hint, but here we are giving more details about the
error itself. And "user catalog tables" looks good to me too (although
existing error messages around RelationIsUsedAsCatalogTable() uses
table used as a catalog table, with "system catalogs" meaning the
core-provided ones. Also, the proposed wording looks consistent with
errdetail_relkind_not_supported(). So, I used that in the attached
0001.

I agree that this is not a hint message. If we ever say something like
"Please do something to get repack to work on user catalog tables,"
that would be a hint, but here we are giving more details about the
error itself.

"User catalog tables" looks good to me too, although the existing
error messages around RelationIsUsedAsCatalogTable() use "table used
as a catalog table," with "system catalogs" meaning the core-provided
ones.

The proposed wording also looks consistent with
errdetail_relkind_not_supported(). So, I used that in the attached
0001.

That said, I also think we can keep the messaging style consistent
within this function. So, in the attached 0002, I changed the existing
error messages for system catalogs, TOAST tables, non-permanent tables
etc. to use detail messages instead of hints, with consistent wording.

Please find the attached v5 patches.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_9
v5-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchapplication/octet-stream; name=v5-0001-Restrict-REPACK-CONCURRENTLY-on-user-catalog-tabl.patchDownload+32-1
v5-0002-Use-detail-messages-consistently-in-REPACK-CONCUR.patchapplication/octet-stream; name=v5-0002-Use-detail-messages-consistently-in-REPACK-CONCUR.patchDownload+14-21
#10Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Alvaro Herrera (#8)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

On Tue, Sep 8, 2026 at 5:47 AM Alvaro Herrera <alvherre@kurilemu.de> wrote:

Hello,

On 2026-Sep-04, Masahiko Sawada wrote:

I reviewed the patch. It's simple and I agree to add a check for user
catalog tables. I've slightly rephrased the comment in
check_concurrent_repack_requirements() to clarify the reason why
REPACK (CONCURRENTLY) doesn't support user catalog tables while it
does for normal tables even though it's a MVCC-unsafe operation. Also
updated the commit message as well. Please review it.

Thanks, I gave this a look and I think it's okay. I'm not sure about
the exact wording of the error message though,

+             ereport(ERROR,
+                             errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+                             errmsg("cannot execute %s on relation \"%s\"",
+                                        "REPACK (CONCURRENTLY)", RelationGetRelationName(rel)),
+                             errhint("%s is not supported for tables used as catalog tables.",
+                                             "REPACK (CONCURRENTLY)"));

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
HINT: REPACK (CONCURRENTLY) is not supported for tables used as catalog tables.

I think the HINT should really be DETAIL, and I'm not sure we need to
mention the command again there. On the other hand, I'm not really
happy with the term "tables used as catalog tables". In the
documentation we use the term "user provided catalog tables" or simply
"user catalog tables" (see <sect2 id="logicaldecoding-capabilities">).
So how about something like this?

ERROR: cannot execute REPACK (CONCURRENTLY) on relation "foobar"
DETAIL: This operation is not supported for user catalog tables.

Looks good to me.

Would you push this patch, or do you want me to?

I can take it if you are okay, as it's a trivial fix.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#11Alvaro Herrera
alvherre@2ndquadrant.com
In reply to: Masahiko Sawada (#10)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

On 2026-Sep-08, Masahiko Sawada wrote:

On Tue, Sep 8, 2026 at 5:47 AM Alvaro Herrera <alvherre@kurilemu.de> wrote:

Would you push this patch, or do you want me to?

I can take it if you are okay, as it's a trivial fix.

Yeah, sounds good, thanks.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"No nos atrevemos a muchas cosas porque son difíciles,
pero son difíciles porque no nos atrevemos a hacerlas" (Séneca)

#12Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Alvaro Herrera (#11)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

On Tue, Sep 8, 2026 at 11:57 AM Alvaro Herrera <alvherre@kurilemu.de> wrote:

On 2026-Sep-08, Masahiko Sawada wrote:

On Tue, Sep 8, 2026 at 5:47 AM Alvaro Herrera <alvherre@kurilemu.de> wrote:

Would you push this patch, or do you want me to?

I can take it if you are okay, as it's a trivial fix.

Yeah, sounds good, thanks.

Pushed.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

#13Bharath Rupireddy
bharath.rupireddyforpostgres@gmail.com
In reply to: Masahiko Sawada (#12)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

Hi,

On Tue, Sep 8, 2026 at 1:09 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Pushed.

Thanks for taking care of this. Please find another remaining patch
rebased to use detail instead of hints and reword the messages
consistently in REPACK (CONCURRENTLY) checks.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com

Attachments:

t253571_13
v6-0001-Use-detail-messages-consistently-in-REPACK-CONCUR.patchapplication/octet-stream; name=v6-0001-Use-detail-messages-consistently-in-REPACK-CONCUR.patchDownload+14-21
#14Masahiko Sawada
sawada.mshk@gmail.com
In reply to: Bharath Rupireddy (#13)
Re: REPACK (CONCURRENTLY) rewrites tables marked with user_catalog_table

On Wed, Sep 9, 2026 at 10:28 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:

Hi,

On Tue, Sep 8, 2026 at 1:09 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:

Pushed.

Thanks for taking care of this. Please find another remaining patch
rebased to use detail instead of hints and reword the messages
consistently in REPACK (CONCURRENTLY) checks.

Thank you for rebasing the patch! Pushed.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com