Bug in asynchronous Append
Hi!
ExecReScanAppend() unconditionally resets callback_pending for all
AsyncRequests. The problem is that postgres_fdw keeps its own knowledge
for the same fact: PgFdwConnState.pendingAreq – a pointer to "pending async
request" for a given connection. That connection can be shared by several
partitions/foreign tables (postgres_fdw caches one connection per
server+usermapping pair). The blind reset in nodeAppend.c only touches the
local AsyncRequest.callback_pending; it never touches
PgFdwConnState.pendingAreq, which correctly points to the still-dangling
request.
Later, when another partition sharing that same connection gets its own
ReScan (for instance, its chgParam changed because of the LATERAL
parameter, and it already has a cursor open), it sends "CLOSE cursor" via
pgfdw_exec_query(). Before sending any new command on the connection, that
function first drains whatever request is still outstanding on it:
if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);
And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix. The
fix postpones the reset of the callback_pending flag
to ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
along with ExecReScan(), which completes the async fetch.
------
Regards,
Alexander Korotkov
Supabase
Attachments:
v1-0001-Fix-corruption-of-async-request-state-on-Append-r.patchapplication/octet-stream; name=v1-0001-Fix-corruption-of-async-request-state-on-Append-r.patchDownload+61-3
Hi Alexander,
On Sat, Jul 4, 2026 at 7:00 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
ExecReScanAppend() unconditionally resets callback_pending for all AsyncRequests. The problem is that postgres_fdw keeps its own knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer to "pending async request" for a given connection. That connection can be shared by several partitions/foreign tables (postgres_fdw caches one connection per server+usermapping pair). The blind reset in nodeAppend.c only touches the local AsyncRequest.callback_pending; it never touches PgFdwConnState.pendingAreq, which correctly points to the still-dangling request.
Later, when another partition sharing that same connection gets its own ReScan (for instance, its chgParam changed because of the LATERAL parameter, and it already has a cursor open), it sends "CLOSE cursor" via pgfdw_exec_query(). Before sending any new command on the connection, that function first drains whatever request is still outstanding on it:
if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix. The fix postpones the reset of the callback_pending flag to ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup along with ExecReScan(), which completes the async fetch.
Interesting! Thanks for the report and patch! Will review.
Best regards,
Etsuro Fujita
Alexander Korotkov писал(а) 2026-07-04 01:00:
Hi!
ExecReScanAppend() unconditionally resets callback_pending for all
AsyncRequests. The problem is that postgres_fdw keeps its own
knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer
to "pending async request" for a given connection. That connection
can be shared by several partitions/foreign tables (postgres_fdw
caches one connection per server+usermapping pair). The blind reset in
nodeAppend.c only touches the local AsyncRequest.callback_pending; it
never touches PgFdwConnState.pendingAreq, which correctly points to
the still-dangling request.Later, when another partition sharing that same connection gets its
own ReScan (for instance, its chgParam changed because of the LATERAL
parameter, and it already has a cursor open), it sends "CLOSE cursor"
via pgfdw_exec_query(). Before sending any new command on the
connection, that function first drains whatever request is still
outstanding on it:if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix.
The fix postpones the reset of the callback_pending flag to
ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
along with ExecReScan(), which completes the async fetch.
Hi. The analysis seems correct to me as well as fix.
--
Best regards,
Alexander Pyhalov,
Postgres Professional
Alexander Korotkov писал(а) 2026-07-04 01:00:
Hi!
ExecReScanAppend() unconditionally resets callback_pending for all
AsyncRequests. The problem is that postgres_fdw keeps its own
knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer
to "pending async request" for a given connection. That connection
can be shared by several partitions/foreign tables (postgres_fdw
caches one connection per server+usermapping pair). The blind reset in
nodeAppend.c only touches the local AsyncRequest.callback_pending; it
never touches PgFdwConnState.pendingAreq, which correctly points to
the still-dangling request.Later, when another partition sharing that same connection gets its
own ReScan (for instance, its chgParam changed because of the LATERAL
parameter, and it already has a cursor open), it sends "CLOSE cursor"
via pgfdw_exec_query(). Before sending any new command on the
connection, that function first drains whatever request is still
outstanding on it:if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix.
The fix postpones the reset of the callback_pending flag to
ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
along with ExecReScan(), which completes the async fetch.------
Regards,
Alexander Korotkov
Supabase
Hi!
It seems, that there is a major issue with the patch, the draining
doesn't
throw away the result in cases when it is required. Consider this case:
-- Expose stale rows from a pending async request that was valid for a
-- previous rescan but is pruned out for the current one.
CREATE VIEW base_tbl2_slow AS
SELECT t.a, t.b, t.c
FROM base_tbl2 t, LATERAL pg_sleep(0.2);
ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2_slow');
SELECT o.x, s.a
FROM (VALUES (2505), (3505)) o(x),
LATERAL (
SELECT a
FROM async_pt
WHERE a = o.x OR (o.x = 2505 AND a = 1505)
LIMIT 1
) s
ORDER BY o.x;
x | a
------+------
2505 | 1505
3505 | 2505
(2 rows)
1) Append executes two async fscans, one of which is slow
2) The first fscan finishes and the append execution stops due to LIMIT
1
3) Second fscan async request is still pending
4) Next append rescan prunes async_p2 for new outer value
5) Old async request for async_p2 is allowed to drain (receive a
callback)
6) The stale tuple from async_p2 is accepted by Append, no
pruning/filter check
7) We get a row that should be impossible (3505 | 2505) in this query
See full reproducer in the attached patch.
Attachments:
v1-regress.patchtext/x-diff; name=v1-regress.patchDownload+82-2
Hi Gleb, all,
On Fri, Jul 10, 2026 at 7:56 PM Gleb Kashkin <g.kashkin@postgrespro.ru> wrote:
Alexander Korotkov писал(а) 2026-07-04 01:00:
ExecReScanAppend() unconditionally resets callback_pending for all
AsyncRequests. The problem is that postgres_fdw keeps its own
knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer
to "pending async request" for a given connection. That connection
can be shared by several partitions/foreign tables (postgres_fdw
caches one connection per server+usermapping pair). The blind reset in
nodeAppend.c only touches the local AsyncRequest.callback_pending; it
never touches PgFdwConnState.pendingAreq, which correctly points to
the still-dangling request.Later, when another partition sharing that same connection gets its
own ReScan (for instance, its chgParam changed because of the LATERAL
parameter, and it already has a cursor open), it sends "CLOSE cursor"
via pgfdw_exec_query(). Before sending any new command on the
connection, that function first drains whatever request is still
outstanding on it:if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix.
The fix postpones the reset of the callback_pending flag to
ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
along with ExecReScan(), which completes the async fetch.
It seems, that there is a major issue with the patch, the draining
doesn't
throw away the result in cases when it is required. Consider this case:-- Expose stale rows from a pending async request that was valid for a
-- previous rescan but is pruned out for the current one.
CREATE VIEW base_tbl2_slow AS
SELECT t.a, t.b, t.c
FROM base_tbl2 t, LATERAL pg_sleep(0.2);
ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2_slow');
SELECT o.x, s.a
FROM (VALUES (2505), (3505)) o(x),
LATERAL (
SELECT a
FROM async_pt
WHERE a = o.x OR (o.x = 2505 AND a = 1505)
LIMIT 1
) s
ORDER BY o.x;
x | a
------+------
2505 | 1505
3505 | 2505
(2 rows)1) Append executes two async fscans, one of which is slow
2) The first fscan finishes and the append execution stops due to LIMIT
1
3) Second fscan async request is still pending
4) Next append rescan prunes async_p2 for new outer value
5) Old async request for async_p2 is allowed to drain (receive a
callback)
6) The stale tuple from async_p2 is accepted by Append, no
pruning/filter check
7) We get a row that should be impossible (3505 | 2505) in this querySee full reproducer in the attached patch.
Thanks for sharing this counter example!
Alexander, sorry to say this, but the idea of delaying the processing
of async requests still pending until ExecAppendAsyncBegin is
completely wrong; that breaks the correctness, as demonstrated by the
example. To guarantee it, I think we should process such requests
*during* ExecReScanAppend, like the attached. For that, I added a new
function ExecAppendAsyncProcessPending, which wouldn't be efficient in
some cases; I think we could optimize the function, but I couldn't
come up with ideas for doing so in a back-patchable way, so I'd like
to leave that for v20. Will work. (In typical cases where the LIMIT
clause isn't included, all of the async subplans would be drained
until ExecReScanAppend, so I don't think that the function would cause
any noticeable overhead.)
This would be my fault. Thanks to all of you!
Best regards,
Etsuro Fujita
Attachments:
Fix-bug-in-async-append-efujita.patchapplication/octet-stream; name=Fix-bug-in-async-append-efujita.patchDownload+72-1
Hi, Etsuro!
On Wed, Jul 15, 2026 at 2:55 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
Hi Gleb, all,
On Fri, Jul 10, 2026 at 7:56 PM Gleb Kashkin <g.kashkin@postgrespro.ru> wrote:
Alexander Korotkov писал(а) 2026-07-04 01:00:
ExecReScanAppend() unconditionally resets callback_pending for all
AsyncRequests. The problem is that postgres_fdw keeps its own
knowledge for the same fact: PgFdwConnState.pendingAreq – a pointer
to "pending async request" for a given connection. That connection
can be shared by several partitions/foreign tables (postgres_fdw
caches one connection per server+usermapping pair). The blind reset in
nodeAppend.c only touches the local AsyncRequest.callback_pending; it
never touches PgFdwConnState.pendingAreq, which correctly points to
the still-dangling request.Later, when another partition sharing that same connection gets its
own ReScan (for instance, its chgParam changed because of the LATERAL
parameter, and it already has a cursor open), it sends "CLOSE cursor"
via pgfdw_exec_query(). Before sending any new command on the
connection, that function first drains whatever request is still
outstanding on it:if (state && state->pendingAreq)
process_pending_request(state->pendingAreq);And process_pending_request() starts with:
Assert(areq->callback_pending);
– which fails, because the flag was corrupted some rounds earlier.
The attached patch contains both the reproduction case and the fix.
The fix postpones the reset of the callback_pending flag to
ExecAppendAsyncBegin(). ExecAppendAsyncBegin() performs this cleanup
along with ExecReScan(), which completes the async fetch.It seems, that there is a major issue with the patch, the draining
doesn't
throw away the result in cases when it is required. Consider this case:-- Expose stale rows from a pending async request that was valid for a
-- previous rescan but is pruned out for the current one.
CREATE VIEW base_tbl2_slow AS
SELECT t.a, t.b, t.c
FROM base_tbl2 t, LATERAL pg_sleep(0.2);
ALTER FOREIGN TABLE async_p2 OPTIONS (SET table_name 'base_tbl2_slow');
SELECT o.x, s.a
FROM (VALUES (2505), (3505)) o(x),
LATERAL (
SELECT a
FROM async_pt
WHERE a = o.x OR (o.x = 2505 AND a = 1505)
LIMIT 1
) s
ORDER BY o.x;
x | a
------+------
2505 | 1505
3505 | 2505
(2 rows)1) Append executes two async fscans, one of which is slow
2) The first fscan finishes and the append execution stops due to LIMIT
1
3) Second fscan async request is still pending
4) Next append rescan prunes async_p2 for new outer value
5) Old async request for async_p2 is allowed to drain (receive a
callback)
6) The stale tuple from async_p2 is accepted by Append, no
pruning/filter check
7) We get a row that should be impossible (3505 | 2505) in this querySee full reproducer in the attached patch.
Thanks for sharing this counter example!
Alexander, sorry to say this, but the idea of delaying the processing
of async requests still pending until ExecAppendAsyncBegin is
completely wrong; that breaks the correctness, as demonstrated by the
example. To guarantee it, I think we should process such requests
*during* ExecReScanAppend, like the attached. For that, I added a new
function ExecAppendAsyncProcessPending, which wouldn't be efficient in
some cases; I think we could optimize the function, but I couldn't
come up with ideas for doing so in a back-patchable way, so I'd like
to leave that for v20. Will work. (In typical cases where the LIMIT
clause isn't included, all of the async subplans would be drained
until ExecReScanAppend, so I don't think that the function would cause
any noticeable overhead.)This would be my fault. Thanks to all of you!
No worries. Thank you for an update and thank you for your patch.
I'm going to push (and backpatch to PG 14 where async append was
introduced) it if no objections.
------
Regards,
Alexander Korotkov
Supabase
Hi Alexander,
On Sun, Jul 19, 2026 at 1:00 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
No worries. Thank you for an update and thank you for your patch.
I'm going to push (and backpatch to PG 14 where async append was
introduced) it if no objections.
Thanks, but this is my fault, so is it possible for me to do so
myself? Anyway, I think the patch will need a bit more work.
Best regards,
Etsuro Fujita
On Sat, Jul 18, 2026 at 10:06 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
On Sun, Jul 19, 2026 at 1:00 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
No worries. Thank you for an update and thank you for your patch.
I'm going to push (and backpatch to PG 14 where async append was
introduced) it if no objections.Thanks, but this is my fault, so is it possible for me to do so
myself? Anyway, I think the patch will need a bit more work.
Sure, please go ahead.
------
Regards,
Alexander Korotkov
Supabase
On Sun, Jul 19, 2026 at 4:16 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
On Sat, Jul 18, 2026 at 10:06 PM Etsuro Fujita <etsuro.fujita@gmail.com> wrote:
On Sun, Jul 19, 2026 at 1:00 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
No worries. Thank you for an update and thank you for your patch.
I'm going to push (and backpatch to PG 14 where async append was
introduced) it if no objections.Thanks, but this is my fault, so is it possible for me to do so
myself? Anyway, I think the patch will need a bit more work.Sure, please go ahead.
Thanks, will do. I have some other priorities in the coming two
weeks, so I'm planning to work on the patch a bit more and
push/backpatch it early next month in time for the August releases.
Best regards,
Etsuro Fujita
Etsuro Fujita писал(а) 2026-07-18 22:59:
On Sun, Jul 19, 2026 at 4:16 AM Alexander Korotkov
<aekorotkov@gmail.com> wrote:On Sat, Jul 18, 2026 at 10:06 PM Etsuro Fujita
<etsuro.fujita@gmail.com> wrote:On Sun, Jul 19, 2026 at 1:00 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
No worries. Thank you for an update and thank you for your patch.
I'm going to push (and backpatch to PG 14 where async append was
introduced) it if no objections.Thanks, but this is my fault, so is it possible for me to do so
myself? Anyway, I think the patch will need a bit more work.Sure, please go ahead.
Thanks, will do. I have some other priorities in the coming two
weeks, so I'm planning to work on the patch a bit more and
push/backpatch it early next month in time for the August releases.Best regards,
Etsuro Fujita
Hi.
I've looked on the suggested patch more attentively in Async Merge
Append thread[1].
There it's a bit more obvious that ExecReScanAppend() behavior in
ExecAppendAsyncProcessPending()
depends on node->as_syncdone. It either sleeps on latch in
ExecAppendAsyncEventWait() or busy loops.
The first behavior seems to be more appropriate, as we still should wait
for all requests
with set callback_pending to complete. Or perhaps, it's not a big
problem, given that this case should
be rare?
1.
/messages/by-id/59be194c5a409fb9fc9f2031581b8a44@postgrespro.ru
--
Best regards,
Alexander Pyhalov,
Postgres Professional
Hi,
On Mon, Aug 3, 2026 at 6:45 PM Alexander Pyhalov
<a.pyhalov@postgrespro.ru> wrote:
I've looked on the suggested patch more attentively in Async Merge
Append thread[1].
There it's a bit more obvious that ExecReScanAppend() behavior in
ExecAppendAsyncProcessPending()
depends on node->as_syncdone. It either sleeps on latch in
ExecAppendAsyncEventWait() or busy loops.
The first behavior seems to be more appropriate, as we still should wait
for all requests
with set callback_pending to complete. Or perhaps, it's not a big
problem, given that this case should
be rare?
Good catch! Incorporated. Attached is an updated version of the
patch. Other changes are:
* Add CHECK_FOR_INTERRUPTS() to the for loop in ExecAppendAsyncProcessPending.
* Merge the async-state-reset code in ExecReScanAppend into
ExecAppendAsyncProcessPending, for readability, and rename that
function to ExecAppendAsyncReset.
* Remove this bit from postgresReScanForeignScan:
/*
* If the node is async-capable, and an asynchronous fetch for it has
* begun, the asynchronous fetch might not have yet completed. Check if
* the node is async-capable, and an asynchronous fetch for it is still in
* progress; if so, complete the asynchronous fetch before restarting the
* scan.
*/
if (fsstate->async_capable &&
fsstate->conn_state->pendingAreq &&
fsstate->conn_state->pendingAreq->requestee == (PlanState *) node)
fetch_more_data(node);
and instead add an assertion, as it's no longer needed due to the
handling in ExecAppendAsyncReset.
* As a test case causing an issue (infinite loop!) on a
non-assert-enabled build, add (a modified version of) Gleb's test case
as well. (The original test case only causes an assertion failure.)
Best regards,
Etsuro Fujita
Attachments:
Fix-bug-in-async-append-efujita-v2.patchapplication/octet-stream; name=Fix-bug-in-async-append-efujita-v2.patchDownload+153-29
Etsuro Fujita писал(а) 2026-08-03 16:57:
Hi,
On Mon, Aug 3, 2026 at 6:45 PM Alexander Pyhalov
<a.pyhalov@postgrespro.ru> wrote:I've looked on the suggested patch more attentively in Async Merge
Append thread[1].
There it's a bit more obvious that ExecReScanAppend() behavior in
ExecAppendAsyncProcessPending()
depends on node->as_syncdone. It either sleeps on latch in
ExecAppendAsyncEventWait() or busy loops.
The first behavior seems to be more appropriate, as we still should
wait
for all requests
with set callback_pending to complete. Or perhaps, it's not a big
problem, given that this case should
be rare?Good catch! Incorporated. Attached is an updated version of the
patch. Other changes are:* Add CHECK_FOR_INTERRUPTS() to the for loop in
ExecAppendAsyncProcessPending.
* Merge the async-state-reset code in ExecReScanAppend into
ExecAppendAsyncProcessPending, for readability, and rename that
function to ExecAppendAsyncReset.
* Remove this bit from postgresReScanForeignScan:/*
* If the node is async-capable, and an asynchronous fetch for it
has
* begun, the asynchronous fetch might not have yet completed.
Check if
* the node is async-capable, and an asynchronous fetch for it is
still in
* progress; if so, complete the asynchronous fetch before
restarting the
* scan.
*/
if (fsstate->async_capable &&
fsstate->conn_state->pendingAreq &&
fsstate->conn_state->pendingAreq->requestee == (PlanState *)
node)
fetch_more_data(node);and instead add an assertion, as it's no longer needed due to the
handling in ExecAppendAsyncReset.* As a test case causing an issue (infinite loop!) on a
non-assert-enabled build, add (a modified version of) Gleb's test case
as well. (The original test case only causes an assertion failure.)
Hi. Looks good to me.
--
Best regards,
Alexander Pyhalov,
Postgres Professional
Hi,
On Tue, Aug 4, 2026 at 5:35 PM Alexander Pyhalov
<a.pyhalov@postgrespro.ru> wrote:
Etsuro Fujita писал(а) 2026-08-03 16:57:
Attached is an updated version of the
patch.
Hi. Looks good to me.
Cool! I'll push the patch and backpatch it to all supported versions.
Thanks for reviewing!
Best regards,
Etsuro Fujita
Etsuro Fujita писал(а) 2026-08-04 14:26:
Hi,
On Tue, Aug 4, 2026 at 5:35 PM Alexander Pyhalov
<a.pyhalov@postgrespro.ru> wrote:Etsuro Fujita писал(а) 2026-08-03 16:57:
Attached is an updated version of the
patch.Hi. Looks good to me.
Cool! I'll push the patch and backpatch it to all supported versions.
Thanks for reviewing!
Best regards,
Etsuro Fujita
Hi. It seems there is the last small issue with the patch.
The following scenario is possible:
1) During rescan, one child of the Append has already produced a tuple
and set as_needrequest
2) Another child still has callback_pending, but its postgres_fdw
connection is occupied by an async request belonging to a different
Append
3) While draining the second child, postgresForeignAsyncConfigureWait()
sees the different requestor and the stale nonempty as_needrequest, so
it returns without registering an event
4) The reset loop then repeats without making progress
See reproducer and fix in the patch. It should be applied to v2.
--
Best regards,
Gleb Kashkin,
Postgres Professional
Gleb Kashkin писал(а) 2026-08-04 15:19:
Etsuro Fujita писал(а) 2026-08-04 14:26:
Hi,
On Tue, Aug 4, 2026 at 5:35 PM Alexander Pyhalov
<a.pyhalov@postgrespro.ru> wrote:Etsuro Fujita писал(а) 2026-08-03 16:57:
Attached is an updated version of the
patch.Hi. Looks good to me.
Cool! I'll push the patch and backpatch it to all supported versions.
Thanks for reviewing!
Best regards,
Etsuro FujitaHi. It seems there is the last small issue with the patch.
The following scenario is possible:
1) During rescan, one child of the Append has already produced a tuple
and set as_needrequest
2) Another child still has callback_pending, but its postgres_fdw
connection is occupied by an async request belonging to a different
Append
3) While draining the second child, postgresForeignAsyncConfigureWait()
sees the different requestor and the stale nonempty as_needrequest, so
it returns without registering an event
4) The reset loop then repeats without making progressSee reproducer and fix in the patch. It should be applied to v2.
Sorry, misclicked. Here is the patch.
--
Best regards,
Gleb Kashkin,
Postgres Professional
Attachments:
v2-regress.patchtext/x-diff; name=v2-regress.patchDownload+82-3
Hi,
On Tue, Aug 4, 2026 at 9:20 PM Gleb Kashkin <g.kashkin@postgrespro.ru> wrote:
Hi. It seems there is the last small issue with the patch.
The following scenario is possible:
1) During rescan, one child of the Append has already produced a tuple
and set as_needrequest
2) Another child still has callback_pending, but its postgres_fdw
connection is occupied by an async request belonging to a different
Append
3) While draining the second child, postgresForeignAsyncConfigureWait()
sees the different requestor and the stale nonempty as_needrequest, so
it returns without registering an event
4) The reset loop then repeats without making progressSee reproducer and fix in the patch. It should be applied to v2.
Sorry, misclicked. Here is the patch.
Good catch! I think that the scenario is possible, and the fix is
correct. I couldn't reproduce it in my environment, though. I think
it probably depends on the environment. I don't want to further
increase the elapsed time for the postgres_fdw regression test, for
this rather-minor case, so sorry, I incorporated only the fix, then
added asserts and tweaked the comment a bit. Attached is a new
version of the patch.
Thanks for the feedback and patch!
Best regards,
Etsuro Fujita
Attachments:
Fix-bug-in-async-append-efujita-v3.patchapplication/octet-stream; name=Fix-bug-in-async-append-efujita-v3.patchDownload+162-29
Etsuro Fujita писал(а) 2026-08-05 15:15:
Thanks for the feedback and patch!
Best regards,
Etsuro Fujita
Thank you for the fix!
--
Best regards,
Gleb Kashkin,
Postgres Professional
On Wed, Aug 5, 2026 at 11:48 PM Gleb Kashkin <g.kashkin@postgrespro.ru> wrote:
Thank you for the fix!
You are welcome! Pushed/backpatched after tweaking a comment a bit.
While working on the back-patch for v14, I noticed that async is
disabled for the test cases in v14. I didn't look into that in
detail, but I think that the reason is probably is_async_capable_plan
in v14, which is restrictive compared to new versions. However, I
think that v14 too would have the same issue, so I applied the patch
to v14 as well after just removing the test cases. I think we could
add test cases for v14 later if needed.
For the record: here is a test case that produces incorrect results on
a production build without the patch. It might depend on the
environment, though. (On an assert-enabled build it causes an
assertion failure.) I didn't add this, though, as it's
time-consuming:
create table base_tbl1 (a int, b int, c text);
create table base_tbl2 (a int, b int, c text);
create table base_tbl3 (a int, b int, c text);
insert into base_tbl1 select 1000 + i, i, to_char(i, 'FM0000') from
generate_series(0, 999, 5) i;
insert into base_tbl2 values (2000, 0, '0000');
insert into base_tbl3 select 3000 + i, i, to_char(i, 'FM0000') from
generate_series(0, 999, 5) i;
create view base_tbl2_slow as with delay as materialized (select
pg_sleep(1.0)) select t.* from base_tbl2 t, delay where t.a > 2500;;
create server loopback foreign data wrapper postgres_fdw options
(dbname 'postgres');
create server loopback2 foreign data wrapper postgres_fdw options
(dbname 'postgres');
alter server loopback options (add async_capable 'true');
alter server loopback2 options (add async_capable 'true');
create user mapping for current_user server loopback;
create user mapping for current_user server loopback2;
create table async_pt (a int, b int, c text) partition by range (a);
create foreign table async_p1 partition of async_pt for values from
(1000) to (2000) server loopback options (table_name 'base_tbl1');
create foreign table async_p2 partition of async_pt for values from
(2000) to (3000) server loopback2 options (table_name
'base_tbl2_slow');
create foreign table async_p3 partition of async_pt for values from
(3000) to (4000) server loopback2 options (table_name 'base_tbl3');
analyze async_pt;
create function rescantest(int) returns boolean as 'begin if $1 < 3000
then return true; else perform count(*) from async_p2; return false;
end if; end;' language plpgsql;
select o.x from (values (2505), (3505)) o(x), lateral (select a from
async_pt where a = o.x or (a = 1505 and rescantest(o.x)) limit 1) s
order by o.x;
Thanks again!
Best regards,
Etsuro Fujita