REPACK (CONCURRENTLY) doesn't check the table AM
REPACK (CONCURRENTLY) replays changes by decoding them from WAL, so it
needs the table AM to be logically decodable, but
check_concurrent_repack_requirements() doesn't check that. Should it be
restricted to heap for v19?
--
nathan
Hi,
On Thu, Aug 27, 2026 at 9:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
REPACK (CONCURRENTLY) replays changes by decoding them from WAL, so it
needs the table AM to be logically decodable, but
check_concurrent_repack_requirements() doesn't check that. Should it be
restricted to heap for v19?
Logical decoding doesn't check the table AM type, but goes ahead and
decodes the WAL records (for that matter, none of CLUSTER, VACUUM
FULL, and REPACK have checks on table AM. They hand off at some point
to the table AM layer). I'm not sure if gating it just for concurrent
repack is the right choice. Is it that we want to have it just for
concurrent repack since it's new code with a new logical decoding
plugin and we want some field reports of needing it for other table
AMs?
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
On Thu, Aug 27, 2026 at 11:59:10AM -0700, Bharath Rupireddy wrote:
On Thu, Aug 27, 2026 at 9:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
REPACK (CONCURRENTLY) replays changes by decoding them from WAL, so it
needs the table AM to be logically decodable, but
check_concurrent_repack_requirements() doesn't check that. Should it be
restricted to heap for v19?Logical decoding doesn't check the table AM type, but goes ahead and
decodes the WAL records (for that matter, none of CLUSTER, VACUUM
FULL, and REPACK have checks on table AM. They hand off at some point
to the table AM layer). I'm not sure if gating it just for concurrent
repack is the right choice. Is it that we want to have it just for
concurrent repack since it's new code with a new logical decoding
plugin and we want some field reports of needing it for other table
AMs?
If the other commands are similarly broken, then perhaps this isn't a valid
open item for v19. But IIUC concurrent repack is the only one of the
commands you listed that depends on logical decoding, and if the table AM
doesn't support that, then it won't work properly.
--
nathan
On Thu, Aug 27, 2026 at 12:16 PM Nathan Bossart
<nathandbossart@gmail.com> wrote:
On Thu, Aug 27, 2026 at 11:59:10AM -0700, Bharath Rupireddy wrote:
On Thu, Aug 27, 2026 at 9:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
REPACK (CONCURRENTLY) replays changes by decoding them from WAL, so it
needs the table AM to be logically decodable, but
check_concurrent_repack_requirements() doesn't check that. Should it be
restricted to heap for v19?Logical decoding doesn't check the table AM type, but goes ahead and
decodes the WAL records (for that matter, none of CLUSTER, VACUUM
FULL, and REPACK have checks on table AM. They hand off at some point
to the table AM layer). I'm not sure if gating it just for concurrent
repack is the right choice. Is it that we want to have it just for
concurrent repack since it's new code with a new logical decoding
plugin and we want some field reports of needing it for other table
AMs?If the other commands are similarly broken, then perhaps this isn't a valid
open item for v19. But IIUC concurrent repack is the only one of the
commands you listed that depends on logical decoding, and if the table AM
doesn't support that, then it won't work properly.
If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites. I think we should have the check for it.
One idea would be to have a new table AM callback returning true if
the table AM supports concurrent repack (i.e., its rmgr provides
rm_decode callback), but I think it's not the right time to introduce
a new table AM callback. So restricting it to heap for v19 seems fine
to me.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Aug 27, 2026 at 12:49 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
On Thu, Aug 27, 2026 at 12:16 PM Nathan Bossart
<nathandbossart@gmail.com> wrote:On Thu, Aug 27, 2026 at 11:59:10AM -0700, Bharath Rupireddy wrote:
On Thu, Aug 27, 2026 at 9:23 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
REPACK (CONCURRENTLY) replays changes by decoding them from WAL, so it
needs the table AM to be logically decodable, but
check_concurrent_repack_requirements() doesn't check that. Should it be
restricted to heap for v19?Logical decoding doesn't check the table AM type, but goes ahead and
decodes the WAL records (for that matter, none of CLUSTER, VACUUM
FULL, and REPACK have checks on table AM. They hand off at some point
to the table AM layer). I'm not sure if gating it just for concurrent
repack is the right choice. Is it that we want to have it just for
concurrent repack since it's new code with a new logical decoding
plugin and we want some field reports of needing it for other table
AMs?If the other commands are similarly broken, then perhaps this isn't a valid
open item for v19. But IIUC concurrent repack is the only one of the
commands you listed that depends on logical decoding, and if the table AM
doesn't support that, then it won't work properly.If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites.
Yes, that's correct. I came to the same conclusion.
I think we should have the check for it.
One idea would be to have a new table AM callback returning true if
the table AM supports concurrent repack (i.e., its rmgr provides
rm_decode callback), but I think it's not the right time to introduce
a new table AM callback.
Agreed. Adding a new table AM callback needs consensus and is time
consuming. Perhaps we can try for HEAD based on field reports that
require the concurrent repack feature for non-heap table AMs.
So restricting it to heap for v19 seems fine
to me.
+1. I will send a patch soon.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Aug 27, 2026 at 12:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites.Yes, that's correct. I came to the same conclusion.
I think we should have the check for it.
One idea would be to have a new table AM callback returning true if
the table AM supports concurrent repack (i.e., its rmgr provides
rm_decode callback), but I think it's not the right time to introduce
a new table AM callback.Agreed. Adding a new table AM callback needs consensus and is time
consuming. Perhaps we can try for HEAD based on field reports that
require the concurrent repack feature for non-heap table AMs.So restricting it to heap for v19 seems fine
to me.+1. I will send a patch soon.
Please find the attached patch. I didn't add a test, I don't think we need one.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Aug 27, 2026 at 1:45 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
Hi,
On Thu, Aug 27, 2026 at 12:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites.Yes, that's correct. I came to the same conclusion.
I think we should have the check for it.
One idea would be to have a new table AM callback returning true if
the table AM supports concurrent repack (i.e., its rmgr provides
rm_decode callback), but I think it's not the right time to introduce
a new table AM callback.Agreed. Adding a new table AM callback needs consensus and is time
consuming. Perhaps we can try for HEAD based on field reports that
require the concurrent repack feature for non-heap table AMs.So restricting it to heap for v19 seems fine
to me.+1. I will send a patch soon.
Please find the attached patch. I didn't add a test, I don't think we need one.
Added the limitation to the repack docs. Please find the attached v2
patch. Sorry for the noise.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
Hi,
On Thu, Aug 27, 2026 at 3:51 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
On Thu, Aug 27, 2026 at 12:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites.Yes, that's correct. I came to the same conclusion.
Please find the attached v3 patch with a test added similar to other
restricted error cases. Sorry for the noise.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
On Mon, Aug 31, 2026 at 8:46 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
Hi,
On Thu, Aug 27, 2026 at 3:51 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:On Thu, Aug 27, 2026 at 12:57 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:If the table AM doesn't support logical decoding, concurrent repack
would silently lose some table data as it misses the changes
happened during the rewrites.Yes, that's correct. I came to the same conclusion.
Please find the attached v3 patch with a test added similar to other
restricted error cases. Sorry for the noise.
I've looked at the patch and have one comment:
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
Probably it's better to use GetHeapamTableAmRoutine() instead of using
heap AM oid. It would allow the following use case the patch adds in
the regression tests:
+CREATE ACCESS METHOD heap2 TYPE TABLE HANDLER heap_tableam_handler;
+CREATE TABLE repack_conc_heap2 (i int) USING heap2;
+REPACK (CONCURRENTLY) repack_conc_heap2;
REPACK'ing on such a table technically works fine. It's unlikely to
drop the check so I think we can add the check without regression
tests.
I attached the updated patch. Please review it.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com
Hi,
On Fri, Sep 4, 2026 at 2:05 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
I've looked at the patch and have one comment:
Thanks for reviewing it.
+ if (rel->rd_rel->relam != HEAP_TABLE_AM_OID)
Probably it's better to use GetHeapamTableAmRoutine() instead of using heap AM oid. It would allow the following use case the patch adds in the regression tests: +CREATE ACCESS METHOD heap2 TYPE TABLE HANDLER heap_tableam_handler; +CREATE TABLE repack_conc_heap2 (i int) USING heap2; +REPACK (CONCURRENTLY) repack_conc_heap2;REPACK'ing on such a table technically works fine. It's unlikely to
drop the check so I think we can add the check without regression
tests.
That works for me. If a table AM that just copies the heap AM handler
as-is works with repack, that is fine. If a table AM that copies the
heap AM handler but overwrites one or more functions errors out with
repack, that still serves the purpose. Also, I am fine without a test
for this because we don't have another table AM in the core and all
tests rely on reusing the same heap AM handler.
I attached the updated patch. Please review it.
The v4 patch looks good to me. pgindent and tests are happy.
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
Hi,
On Fri, Sep 4, 2026 at 4:13 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
I attached the updated patch. Please review it.
The v4 patch looks good to me. pgindent and tests are happy.
Please find the attached v5 patch containing detail instead of hint
and also consistent wording as discussed here:
/messages/by-id/CALj2ACXyTPZEcGeKgHQjtSCScOOLpHNhAdH3Cn2T48aXg2LZfA@mail.gmail.com
--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com
Attachments:
t253576_11v5-0001-Restrict-REPACK-CONCURRENTLY-to-the-heap-access-m.patchapplication/octet-stream; name=v5-0001-Restrict-REPACK-CONCURRENTLY-to-the-heap-access-m.patchDownload+20-1
On Tue, Sep 8, 2026 at 10:24 AM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
Hi,
On Fri, Sep 4, 2026 at 4:13 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:I attached the updated patch. Please review it.
The v4 patch looks good to me. pgindent and tests are happy.
Please find the attached v5 patch containing detail instead of hint
and also consistent wording as discussed here:
/messages/by-id/CALj2ACXyTPZEcGeKgHQjtSCScOOLpHNhAdH3Cn2T48aXg2LZfA@mail.gmail.com
Thank you for updating the patch. It looks good to me so I've pushed.
Regards,
--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com