Table AM callback for logical decoding support (REPACK CONCURRENTLY)

Started by Andrew Dunstan3 days ago10 messageshackers
Jump to latest
#1Andrew Dunstan
andrew@dunslane.net

Hi,

In [1]/messages/by-id/apBkixO4xAGFoiT3@nathan, REPACK (CONCURRENTLY) was restricted to heap because nothing in
TableAmRoutine says whether an AM can be logically decoded, and without
decoding the changes made during the rewrite are silently lost. Masahiko
suggested a callback for HEAD, and Bharath that it wait for a field report.
Here is one: a table AM we are developing has its own resource manager
with rm_decode, and the heap-only check refuses it.

The attached patch adds an optional relation_supports_logical_decoding
callback, last in TableAmRoutine.  An AM without it is taken not to support
decoding, so existing AMs are unaffected.  heap provides it, and REPACK asks
the AM instead of comparing its routine with heap's.

With the callback set in that AM, REPACK (CONCURRENTLY) with concurrent DML
converges row for row with a heap oracle, and a logical slot attached
during it sees no duplicate changes.

The test module has two AMs that copy heap's routine, one without the
callback.  heap_getnext() insists on heap's own routine, so their tables can
have no index or TOAST table; the test shows one refused and the other past
the check.

This is for v20 only.  It is the second small TableAmRoutine addition that
out-of-tree AMs need, after the reloptions patch in [2]/messages/by-id/ea1c4d33-0780-473c-96dc-1468cf733a04@dunslane.net     ("Allow table AMs to define their own reloptions").  The two don't
conflict (this one is last in the struct), and could be reviewed, or
committed, as one series if that's easier. Maybe we only need one new test
module instead of two.

I'd welcome views on the shape: a callback, as suggested, or a plain flag.

[1]: /messages/by-id/apBkixO4xAGFoiT3@nathan
[2]: /messages/by-id/ea1c4d33-0780-473c-96dc-1468cf733a04@dunslane.net     ("Allow table AMs to define their own reloptions")
    ("Allow table AMs to define their own reloptions")

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

t259483_1
0001-Add-a-table-AM-callback-for-logical-decoding-support.patchtext/x-patch; charset=UTF-8; name=0001-Add-a-table-AM-callback-for-logical-decoding-support.patchDownload+268-8
#2Chao Li
li.evan.chao@gmail.com
In reply to: Andrew Dunstan (#1)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

On Nov 20, 2026, at 00:34, Andrew Dunstan <andrew@dunslane.net> wrote:

Hi,

In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
TableAmRoutine says whether an AM can be logically decoded, and without
decoding the changes made during the rewrite are silently lost. Masahiko
suggested a callback for HEAD, and Bharath that it wait for a field report.
Here is one: a table AM we are developing has its own resource manager
with rm_decode, and the heap-only check refuses it.

The attached patch adds an optional relation_supports_logical_decoding
callback, last in TableAmRoutine. An AM without it is taken not to support
decoding, so existing AMs are unaffected. heap provides it, and REPACK asks
the AM instead of comparing its routine with heap's.

With the callback set in that AM, REPACK (CONCURRENTLY) with concurrent DML
converges row for row with a heap oracle, and a logical slot attached
during it sees no duplicate changes.

The test module has two AMs that copy heap's routine, one without the
callback. heap_getnext() insists on heap's own routine, so their tables can
have no index or TOAST table; the test shows one refused and the other past
the check.

This is for v20 only. It is the second small TableAmRoutine addition that
out-of-tree AMs need, after the reloptions patch in [2]. The two don't
conflict (this one is last in the struct), and could be reviewed, or
committed, as one series if that's easier. Maybe we only need one new test
module instead of two.

I'd welcome views on the shape: a callback, as suggested, or a plain flag.

[1] /messages/by-id/apBkixO4xAGFoiT3@nathan
[2] /messages/by-id/ea1c4d33-0780-473c-96dc-1468cf733a04@dunslane.net
("Allow table AMs to define their own reloptions")

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com
<0001-Add-a-table-AM-callback-for-logical-decoding-support.patch>

I think it's better to use a table AM method instead of sticking to the heap AM. This seems to be in the same spirit as patch [1]/messages/by-id/DE69B65E-4621-4AF4-9A00-6A042E565C90@gmail.com I'm currently working on, which adds a new TAM method to decide whether a relation is empty.

After reading the patch, I just have a question: Why does relation_supports_logical_decoding() need a Relation argument? From my understanding, if or not an AM supports logical decoding seems to be an AM-level capability, and the heap AM's implementation only returns true. Is there a known use case where a table AM needs to make this decision based on an individual relation?

[1]: /messages/by-id/DE69B65E-4621-4AF4-9A00-6A042E565C90@gmail.com

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

#3Antonin Houska
ah@cybertec.at
In reply to: Andrew Dunstan (#1)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

Andrew Dunstan <andrew@dunslane.net> wrote:

In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
TableAmRoutine says whether an AM can be logically decoded, and without
decoding the changes made during the rewrite are silently lost. Masahiko
suggested a callback for HEAD, and Bharath that it wait for a field report.
Here is one: a table AM we are developing has its own resource manager
with rm_decode, and the heap-only check refuses it.

The attached patch adds an optional relation_supports_logical_decoding
callback, last in TableAmRoutine.  An AM without it is taken not to support
decoding, so existing AMs are unaffected.  heap provides it, and REPACK asks
the AM instead of comparing its routine with heap's.

Have you considered adding a callback / field that provides the AM's RmgrId ?
The AM needs to know it anyway, at least when inserting WAL records (Well, the
heap AM actually uses two ids, RM_HEAP_ID and RM_HEAP2_ID ...). With this
knowledge, REPACK (CONCURRENTLY) could check whether the rm_decode() callback
exists.

--
Antonin Houska
Web: https://www.cybertec-postgresql.com

#4Xuneng Zhou
xunengzhou@gmail.com
In reply to: Andrew Dunstan (#1)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

Hi Andrew,

The test module has two AMs that copy heap's routine, one without the
callback. heap_getnext() insists on heap's own routine, so their tables can
have no index or TOAST table; the test shows one refused and the other past
the check.

A small comment for the patch:

The REPACK(CONCURRENTLY) statements failed the wal-level checking with
wal_level=mininal in my testing. This seems to be the cause of
4b2aa4b39cb to move concurrent REPACK tests out of cluster.sql. Does
it make sense to follow pg_walinspect and set wal_level = replica in a
temp config for make and meson?

--
Regards,
Xuneng Zhou
HighGo Software Co., Ltd.

#5Andrew Dunstan
andrew@dunslane.net
In reply to: Antonin Houska (#3)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

Thanks everyone for looking.

On 2026-11-20 Fr 2:54 AM, Antonin Houska wrote:

Andrew Dunstan <andrew@dunslane.net> wrote:

In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
TableAmRoutine says whether an AM can be logically decoded, and without
decoding the changes made during the rewrite are silently lost. Masahiko
suggested a callback for HEAD, and Bharath that it wait for a field report.
Here is one: a table AM we are developing has its own resource manager
with rm_decode, and the heap-only check refuses it.

The attached patch adds an optional relation_supports_logical_decoding
callback, last in TableAmRoutine.  An AM without it is taken not to support
decoding, so existing AMs are unaffected.  heap provides it, and REPACK asks
the AM instead of comparing its routine with heap's.

Have you considered adding a callback / field that provides the AM's RmgrId ?
The AM needs to know it anyway, at least when inserting WAL records (Well, the
heap AM actually uses two ids, RM_HEAP_ID and RM_HEAP2_ID ...). With this
knowledge, REPACK (CONCURRENTLY) could check whether the rm_decode() callback
exists.

I did think about exposing the RmgrId, and it's appealing because
the AM has to know it anyway.  I ended up not going that way, for two
reasons.

First, an AM doesn't have a single rmgr.  Heap uses two, RM_HEAP_ID and
RM_HEAP2_ID, as you say, so we'd need the AM to hand back a list, or
REPACK would have to know which of them matters.  That's more API than a
yes/no question warrants.

Second, a non-NULL rm_decode tells us the rmgr decodes some of its records.
It doesn't tell us that the AM's changes reach the reorder buffer in a form
REPACK can apply, or that every record type the AM writes during a rewrite
is covered.  The AM is the only one in a position to say that, and the
callback lets it.

Do you see a case where the callback alone isn't enough?

On 2026-11-19 Th 9:40 PM, Chao Li wrote:

After reading the patch, I just have a question: Why does relation_supports_logical_decoding() need a Relation argument? From my understanding, if or not an AM supports logical decoding seems to be an AM-level capability, and the heap AM's implementation only returns true. Is there a known use case where a table AM needs to make this decision based on an individual relation?

I don't have a concrete use case for deciding per relation.  I kept the
Relation argument so that an AM could decide per table, say from a reloption
or a storage mode, without having to change the callback's signature later.

On 2026-11-20 Fr 4:09 AM, Xuneng Zhou wrote:

The REPACK(CONCURRENTLY) statements failed the wal-level checking with
wal_level=mininal in my testing. This seems to be the cause of
4b2aa4b39cb to move concurrent REPACK tests out of cluster.sql. Does
it make sense to follow pg_walinspect and set wal_level = replica in a
temp config for make and meson?

Yes, good catch.  I only ran the test module with the default
wal_level, so I never saw the failure.  With wal_level = minimal, all three
REPACK (CONCURRENTLY) statements fail the wal_level check in
check_concurrent_repack_requirements() before they reach the AM check,
so the
test output is wrong.  v2 follows the pg_walinspect pattern.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

t259483_5
v2-0001-Add-a-table-AM-callback-for-logical-decoding-supp.patchtext/x-patch; charset=UTF-8; name=v2-0001-Add-a-table-AM-callback-for-logical-decoding-supp.patchDownload+279-8
#6Antonin Houska
ah@cybertec.at
In reply to: Andrew Dunstan (#5)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

Andrew Dunstan <andrew@dunslane.net> wrote:

On 2026-11-20 Fr 2:54 AM, Antonin Houska wrote:

Andrew Dunstan <andrew@dunslane.net> wrote:

In [1], REPACK (CONCURRENTLY) was restricted to heap because nothing in
TableAmRoutine says whether an AM can be logically decoded, and without
decoding the changes made during the rewrite are silently lost. Masahiko
suggested a callback for HEAD, and Bharath that it wait for a field report.
Here is one: a table AM we are developing has its own resource manager
with rm_decode, and the heap-only check refuses it.

The attached patch adds an optional relation_supports_logical_decoding
callback, last in TableAmRoutine.  An AM without it is taken not to support
decoding, so existing AMs are unaffected.  heap provides it, and REPACK asks
the AM instead of comparing its routine with heap's.

Have you considered adding a callback / field that provides the AM's RmgrId ?
The AM needs to know it anyway, at least when inserting WAL records (Well, the
heap AM actually uses two ids, RM_HEAP_ID and RM_HEAP2_ID ...). With this
knowledge, REPACK (CONCURRENTLY) could check whether the rm_decode() callback
exists.

I did think about exposing the RmgrId, and it's appealing because
the AM has to know it anyway.  I ended up not going that way, for two
reasons.

First, an AM doesn't have a single rmgr.  Heap uses two, RM_HEAP_ID and
RM_HEAP2_ID, as you say, so we'd need the AM to hand back a list, or
REPACK would have to know which of them matters.  That's more API than a
yes/no question warrants.

ok

Second, a non-NULL rm_decode tells us the rmgr decodes some of its records.
It doesn't tell us that the AM's changes reach the reorder buffer in a form
REPACK can apply, or that every record type the AM writes during a rewrite
is covered.  The AM is the only one in a position to say that, and the
callback lets it.

I supposed that the AM author is familiar with the RMGR, and that in the
typical case both AM and RMGR are included in the same extension.

Do you see a case where the callback alone isn't enough?

(I suppose you mean the table AM callback you propose rather than rm_decode())

No, I don't. I was just thinking whether the new AM callback can be more
generic and thus useful for more purposes. Exactly as you say, the RmgrId
approach is appealing :-)

--
Antonin Houska
Web: https://www.cybertec-postgresql.com

#7Alvaro Herrera
alvherre@kurilemu.de
In reply to: Andrew Dunstan (#5)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

I suppose as we get more and more external table AMs and functionality
we're going to want more and more booleans to say that an AM supports
this or that. Why not do it just once and invent and interface that
says

uint32 (*relation_supports_feature) (uint32)

so that the code can pass a bunch of TABLEAM_SUPPORTS_FOOBAR flags, and
the routine can fill in the return word with 1 for the features it
supports and 0 for the ones it doesn't? Then the calling code can ask
for just one or more than one, without spending time on determining
whether it supports bits it wasn't asked about, in case some of them are
expensive to determine.

Maybe add more arguments to the callback in case we deem they necessary,
say the Relation itself or whatever.

This would save us in the future having to add more and more callbacks
to add for specific individual bits. tableam.h just defines
TABLEAM_SUPPORTS_LOGICAL_DECODING and so forth, heapam knows to set that
to 1.

--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
"Siempre hay que alimentar a los dioses, aunque la tierra esté seca" (Orual)

#8Andrew Dunstan
andrew@dunslane.net
In reply to: Alvaro Herrera (#7)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

On 2026-11-20 Fr 3:04 PM, Álvaro Herrera wrote:

I suppose as we get more and more external table AMs and functionality
we're going to want more and more booleans to say that an AM supports
this or that. Why not do it just once and invent and interface that
says

uint32 (*relation_supports_feature) (uint32)

so that the code can pass a bunch of TABLEAM_SUPPORTS_FOOBAR flags, and
the routine can fill in the return word with 1 for the features it
supports and 0 for the ones it doesn't? Then the calling code can ask
for just one or more than one, without spending time on determining
whether it supports bits it wasn't asked about, in case some of them are
expensive to determine.

Maybe add more arguments to the callback in case we deem they necessary,
say the Relation itself or whatever.

This would save us in the future having to add more and more callbacks
to add for specific individual bits. tableam.h just defines
TABLEAM_SUPPORTS_LOGICAL_DECODING and so forth, heapam knows to set that
to 1.

That seems reasonable. v3 attached.

The callback is now

    uint32 (*relation_supports_features) (Relation rel, uint32 features);

TABLEAM_SUPPORTS_LOGICAL_DECODING is the only flag so far.  It is passed
the flags the caller is asking about and returns the ones the relation
supports.  An AM without the callback supports none, and so does one that
predates a flag, so existing AMs are unaffected and a new flag cannot change
what an old AM claims.  The callback isn't expected to work out bits that
weren't asked for, as you suggested; table_relation_supports_features()
also masks the result with the request, so an AM that over-reports can't
confuse a caller.  There is a single-flag wrapper,
table_relation_supports_feature(), which is what REPACK calls.

I've kept the scope to capabilities of the AM.  The "is this relation
empty" callback Chao is working on asks about the data, not about what the
AM can do, so I don't think it belongs in the feature word.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

t259483_8
v3-0001-Add-a-table-AM-callback-to-report-supported-featu.patchtext/x-patch; charset=UTF-8; name=v3-0001-Add-a-table-AM-callback-to-report-supported-featu.patchDownload+399-8
#9Zsolt Parragi
zsolt.parragi@percona.com
In reply to: Andrew Dunstan (#8)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)
+ * TABLEAM_SUPPORTS_LOGICAL_DECODING: every change to the relation's rows can
+ * be logically decoded, which needs the AM's resource manager to have an
+ * rm_decode callback.  REPACK (CONCURRENTLY) checks this, as it would
+ * otherwise lose the changes made while it runs.

Doesn't it also require filtering (change_useless_for_repack)?

Also, the helper can crash with partitioned tables or views:

CREATE TABLE p (i int PRIMARY KEY, j int) PARTITION BY RANGE (i);
SELECT test_tableam_supports_features('p'::regclass, 1);
server closed the connection unexpectedly
LOG: client backend (PID 965976) was terminated by signal 11: Segmentation fault
LOG: all server processes terminated; reinitializing

#10Andrew Dunstan
andrew@dunslane.net
In reply to: Zsolt Parragi (#9)
Re: Table AM callback for logical decoding support (REPACK CONCURRENTLY)

On 2026-11-20 Fr 7:00 PM, Zsolt Parragi wrote:

+ * TABLEAM_SUPPORTS_LOGICAL_DECODING: every change to the relation's rows can
+ * be logically decoded, which needs the AM's resource manager to have an
+ * rm_decode callback.  REPACK (CONCURRENTLY) checks this, as it would
+ * otherwise lose the changes made while it runs.

Doesn't it also require filtering (change_useless_for_repack)?

Yes, it does, and the comment should have said so.  pgrepack's change
callback asserts that it only sees changes to the relation being rebuilt
("Changes of other relation should not have been decoded"), so an AM whose
rm_decode doesn't call change_useless_for_repack() would trip that on an
assert build, and on a non-assert build feed other tables' changes into the
rewrite.  Our AM's rm_decode already does it for each kind of change, which
is how I missed that the contract didn't mention it. I have added it to the
comment on the flag and to tableam.sgml.

It's also another reason the RmgrId idea wouldn't have been enough:
knowing that a resource manager has an rm_decode tells us nothing about
whether it applies the filter.

Also, the helper can crash with partitioned tables or views:

CREATE TABLE p (i int PRIMARY KEY, j int) PARTITION BY RANGE (i);
SELECT test_tableam_supports_features('p'::regclass, 1);
server closed the connection unexpectedly
LOG: client backend (PID 965976) was terminated by signal 11:
Segmentation fault
LOG: all server processes terminated; reinitializing

Thanks, I can reproduce that: a partitioned table or a view has no table AM,
so test_tableam_supports_features() dereferenced a NULL rd_tableam. 
It's only
the test helper, since REPACK checks the relkind first, but it was
wrong.  The
helper now rejects anything without a table AM ("is not a table"), and the
test module covers a partitioned table, a view and an index.  I also
added an
Assert on rd_tableam to table_relation_supports_features(), so misuse fails
clearly on an assert build; I didn't add a guard there, as the other table_*
wrappers don't have one.

v4 is attached.  It passes the regress, isolation, test_decoding and
injection_points suites.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Attachments:

t259483_10
v4-0001-Add-a-table-AM-callback-to-report-supported-featu.patchtext/x-patch; charset=UTF-8; name=v4-0001-Add-a-table-AM-callback-to-report-supported-featu.patchDownload+440-8