Fix ReadStream handling when heap rescan strategy changes

Started by Bertrand Drouvot19 days ago12 messageshackers
Jump to latest
#1Bertrand Drouvot
bertranddrouvot.pg@gmail.com

Hi hackers,

While working on [1]/messages/by-id/auCYYbDyrQEwIsQq@bdtpg, I noticed that a heap rescan can change whether a buffer
access strategy is used while retaining the existing ReadStream. If the strategy
is disabled, the stream keeps a dangling pointer after it is freed. If it is
enabled, the stream continues using its NULL strategy and the queue capacity
selected at creation.

The issue was introduced by b7b0f3f27241. Nazir pointed it out and Yuhang
proposed fixing it with a new read_stream_set_strategy() API in [2]/messages/by-id/CAN55FZ3Lik2m9J+EQjzEHSMSBNgAXU5Myt8LFg=D2OQ0i_jUMQ@mail.gmail.com. Since the
issue is independent of the streaming SampleScan changes, I wonder if it wouldn't
make sense to fix it separately, as in the attached?

0001 takes a different approach (than [2]/messages/by-id/CAN55FZ3Lik2m9J+EQjzEHSMSBNgAXU5Myt8LFg=D2OQ0i_jUMQ@mail.gmail.com) and recreates the ReadStream when
initscan() changes whether a strategy is active. Ordinary rescans continue to
reset and reuse the existing stream. It uses the existing ReadStream APIs, which
could also make the backpatches easier.

0002 contains a test. I'm not sure it is worth adding to the tree given the amount
of setup, but it demonstrates the issue.

[1]: /messages/by-id/auCYYbDyrQEwIsQq@bdtpg
[2]: /messages/by-id/CAN55FZ3Lik2m9J+EQjzEHSMSBNgAXU5Myt8LFg=D2OQ0i_jUMQ@mail.gmail.com

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Attachments:

t254538_1
v1-0001-Fix-ReadStream-handling-when-heap-rescan-strategy.patchtext/x-diff; charset=us-asciiDownload+76-47
v1-0002-Test-heap-rescan-strategy-changes.patchtext/x-diff; charset=us-asciiDownload+82-1
#2Nazir Bilal Yavuz
byavuz81@gmail.com
In reply to: Bertrand Drouvot (#1)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi,

On Tue, 3 Nov 2026 at 09:41, Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

While working on [1], I noticed that a heap rescan can change whether a buffer
access strategy is used while retaining the existing ReadStream. If the strategy
is disabled, the stream keeps a dangling pointer after it is freed. If it is
enabled, the stream continues using its NULL strategy and the queue capacity
selected at creation.

The issue was introduced by b7b0f3f27241. Nazir pointed it out and Yuhang
proposed fixing it with a new read_stream_set_strategy() API in [2]. Since the
issue is independent of the streaming SampleScan changes, I wonder if it wouldn't
make sense to fix it separately, as in the attached?

Fixing it separately makes sense to me.

0001 takes a different approach (than [2]) and recreates the ReadStream when
initscan() changes whether a strategy is active. Ordinary rescans continue to
reset and reuse the existing stream. It uses the existing ReadStream APIs, which
could also make the backpatches easier.

The main difference between Yuhang's ([2]updates the existing stream through read_stream_set_strategy() when the strategy is enabled but your approach ends and re-creates the stream. Both approaches end and re-create the stream if the strategy is disabled. It seems [2]'s performance is better because of this reason but I am not sure whether it makes a measurable difference.) and your approach is that
[2]: updates the existing stream through read_stream_set_strategy() when the strategy is enabled but your approach ends and re-creates the stream. Both approaches end and re-create the stream if the strategy is disabled. It seems [2]'s performance is better because of this reason but I am not sure whether it makes a measurable difference.
when the strategy is enabled but your approach ends and re-creates the
stream. Both approaches end and re-create the stream if the strategy
is disabled. It seems [2]updates the existing stream through read_stream_set_strategy() when the strategy is enabled but your approach ends and re-creates the stream. Both approaches end and re-create the stream if the strategy is disabled. It seems [2]'s performance is better because of this reason but I am not sure whether it makes a measurable difference.'s performance is better because of this
reason but I am not sure whether it makes a measurable difference.

Although I didn't find a correctness problem with [2]updates the existing stream through read_stream_set_strategy() when the strategy is enabled but your approach ends and re-creates the stream. Both approaches end and re-create the stream if the strategy is disabled. It seems [2]'s performance is better because of this reason but I am not sure whether it makes a measurable difference. during my
earlier review, I can see the simplicity benefit here. Recreating the
stream keeps strategy-dependent initialization in one place. That
seems a reasonable approach for backpatching.

0002 contains a test. I'm not sure it is worth adding to the tree given the amount
of setup, but it demonstrates the issue.

I haven't reviewed 0002 yet, but I think regression coverage for this
bug would be worthwhile.

Minor readability suggestion:

+ if (scan->rs_read_stream && allow_strat != (scan->rs_strategy != NULL))

We can replace both '!=' with '=='. I find it slightly easier to read,
just a suggestion.

--
Regards,
Nazir Bilal Yavuz
Microsoft

#3shihao zhong
zhong950419@gmail.com
In reply to: Nazir Bilal Yavuz (#2)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi Bertrand,

I am not sure the dangling pointer case can happen on master today.

initscan() only frees the strategy if SO_ALLOW_STRAT gets cleared or
the table drops below NBuffers / 4. The only caller of
table_rescan_set_params() is SampleScan, and a sample scan has no read
stream on master.

I don't think the table can shrink either, because truncation takes an AE
today.
This is something I really hope we can get fixed/improved.

Back to the main topic, I think on master the worst case scenario is the
one in your 0002, which causes more buffer pollution.

Thanks,
Shihao

#4Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: shihao zhong (#3)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi Shihao,

On Tue, Nov 03, 2026 at 10:32:25PM -0500, shihao zhong wrote:

Hi Bertrand,

I am not sure the dangling pointer case can happen on master today.

initscan() only frees the strategy if SO_ALLOW_STRAT gets cleared or
the table drops below NBuffers / 4. The only caller of
table_rescan_set_params() is SampleScan, and a sample scan has no read
stream on master.

I don't think the table can shrink either, because truncation takes an AE
today.

Thanks for looking at it!

I think that an extension can trigger it without shrinking the relation: start
a large sequential scan with allow_strat = true, then call table_rescan_set_params()
with allow_strat = false.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#5Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Nazir Bilal Yavuz (#2)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi,

On Tue, Nov 03, 2026 at 01:23:26PM +0300, Nazir Bilal Yavuz wrote:

Hi,

On Tue, 3 Nov 2026 at 09:41, Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

The issue was introduced by b7b0f3f27241. Nazir pointed it out and Yuhang
proposed fixing it with a new read_stream_set_strategy() API in [2]. Since the
issue is independent of the streaming SampleScan changes, I wonder if it wouldn't
make sense to fix it separately, as in the attached?

Fixing it separately makes sense to me.

Thanks for looking at it!

The main difference between Yuhang's ([2]) and your approach is that
[2] updates the existing stream through read_stream_set_strategy()
when the strategy is enabled but your approach ends and re-creates the
stream. Both approaches end and re-create the stream if the strategy
is disabled. It seems [2]'s performance is better because of this
reason but I am not sure whether it makes a measurable difference.

OTOH, [2] calls read_stream_set_strategy() on every rescan when the bulk strategy
remains enabled. FWIW, I ran a few benchmarks, and regardless of which approach
was faster, the absolute difference remained below 0.15 microseconds per rescan.
Each measurement timed 3 million rescans, and the results were repeatable across
15 samples.

So I don't think performance strongly favors either approach.

Although I didn't find a correctness problem with [2] during my
earlier review, I can see the simplicity benefit here. Recreating the
stream keeps strategy-dependent initialization in one place. That
seems a reasonable approach for backpatching.

+1

0002 contains a test. I'm not sure it is worth adding to the tree given the amount
of setup, but it demonstrates the issue.

I haven't reviewed 0002 yet, but I think regression coverage for this
bug would be worthwhile.

Yeah, I think the test is worthwhile. It initializes and starts a new cluster,
which is not that cheap (and our test code has been growing quite a bit lately)
but that seems justified here.

Minor readability suggestion:

+ if (scan->rs_read_stream && allow_strat != (scan->rs_strategy != NULL))

We can replace both '!=' with '=='. I find it slightly easier to read,
just a suggestion.

Yeah, that's probably just a matter of taste. I find the existing links more
directly to "the requested state differs from the current state" but I'm happy
to change it if you feel strongly about it (I don't).

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#6shihao zhong
zhong950419@gmail.com
In reply to: Bertrand Drouvot (#4)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi Bertrand,

I think that an extension can trigger it without shrinking the relation

I think you are right here and I missed that case. Not sure if any extension
really did it or not.

So my earlier mail was too strong. Core can't reach it, but an
extension can. It might be worth saying that in the commit message.

Thanks,
Shihao

#7Joao Rito
joaovictor.rito@gmail.com
In reply to: shihao zhong (#6)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi Bertrand,

Thank you for the patch. I tested v1 (0001 and 0002) and it looks good to
me.

What I did:

- Applied both patches to current master (165af118c5). They apply cleanly
and build without warnings (--enable-cassert --enable-debug
--enable-tap-tests).
- make check passes (243 tests). The whole src/test/modules/test_misc
suite passes with the patch (16 files, 169 tests), including the new
t/016_heap_rescan_strategy.pl.
- I also ran the new test against unpatched master. It fails there on
"a rescanned sequential scan uses its newly selected bulk-read strategy"
(got 0, expected > 0). So the test does catch the bug, which I think
helps with the question of whether it is worth its setup cost.
- I ran the query from the test by hand (shared_buffers = 1MB,
max_parallel_workers_per_gather = 0) and looked at
pg_stat_get_backend_io() for the bulkread context:
- master: 0 reads, 0 hits in bulkread, with io_method = worker and
io_method = sync.
- patched: 468 reads, 104 hits in bulkread, with both io_method values.
The query returns the same 11997 rows in both builds, and there were no
assertion failures or errors in the server log.

One small thing: the commit message of 0001 has an empty "Reviewed-by:"
and "Discussion: https://postgr.es/m/" without the message-id. I guess you
will fill those in later.

I only tested the case where the rescan starts using a strategy. I did not
exercise the other direction (the strategy is disabled on rescan, the
extension case you described), and I did not test parallel, bitmap or TID
range scans, io_uring, the back branches, or a meson build.

Thanks again,

Joao Rito

#8Melanie Plageman
melanieplageman@gmail.com
In reply to: Bertrand Drouvot (#1)
Re: Fix ReadStream handling when heap rescan strategy changes

On Tue, Nov 3, 2026 at 1:40 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

0001 takes a different approach (than [2]) and recreates the ReadStream when
initscan() changes whether a strategy is active. Ordinary rescans continue to
reset and reuse the existing stream. It uses the existing ReadStream APIs, which
could also make the backpatches easier.

Now that initscan() has to concern itself with the read stream because
it has to reset or end the stream before changing the strategy, then
change the strategy, and then make the new read stream, I think the
separation between initscan() and making the read stream is no longer
useful. In fact, the ping-ponging has become confusing. I think your
version might actually have read_stream_reset() getting called twice
in some cases since read_stream_end() also calls it. Attached v2 just
moves all the read stream setup into initscan() and adds some flags to
make it more clear when we want to rebuild the stream/strategy.

0002 contains a test. I'm not sure it is worth adding to the tree given the amount
of setup, but it demonstrates the issue.

I agree that zero test coverage of this is not good. I really would
have liked to find somewhere to put the test that wasn't a new TAP
test file that makes yet another new cluster. Alas, neither I nor my
LLM could come up with a good place. I did however, change the test to
use a cursor instead of a query that does a rescan. That takes away
the need for the plpgsql function and makes the test simpler IMO.
pg_stat_get_backend_io() doesn't exist in 17, so we'll have to modify
the test to backpatch there.

- Melanie

Attachments:

t254538_8
v2-0001-Recreate-read-stream-across-rescans.patchtext/x-patch; charset=US-ASCII; name=v2-0001-Recreate-read-stream-across-rescans.patchDownload+142-73
#9Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Melanie Plageman (#8)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi,

On Tue, Nov 17, 2026 at 04:54:08PM -0500, Melanie Plageman wrote:

On Tue, Nov 3, 2026 at 1:40 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

0001 takes a different approach (than [2]) and recreates the ReadStream when
initscan() changes whether a strategy is active. Ordinary rescans continue to
reset and reuse the existing stream. It uses the existing ReadStream APIs, which
could also make the backpatches easier.

Now that initscan() has to concern itself with the read stream because
it has to reset or end the stream before changing the strategy, then
change the strategy, and then make the new read stream, I think the
separation between initscan() and making the read stream is no longer
useful. In fact, the ping-ponging has become confusing. I think your
version might actually have read_stream_reset() getting called twice
in some cases since read_stream_end() also calls it. Attached v2 just
moves all the read stream setup into initscan() and adds some flags to
make it more clear when we want to rebuild the stream/strategy.

Thanks for looking at it!

Yeah, that makes sense to me. Having initscan() own the reset/end, strategy update,
and stream creation makes the ordering clearer (and also avoids the possible double
reset).

I've one Nit, if we want this comment to describe the complete stream lifecycle,
in heapam.h:

"
stream is allocated at the beginning of the scan and reset on rescan or
when the scan direction changes.
"

worth to change it to something like?

"
The read stream is allocated at the beginning of the scan. It is reset when
the scan direction changes. On rescan, it is reset if strategy use remains
unchanged, and recreated otherwise.
"

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#10Bertrand Drouvot
bertranddrouvot.pg@gmail.com
In reply to: Melanie Plageman (#8)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi,

On Tue, Nov 17, 2026 at 04:54:08PM -0500, Melanie Plageman wrote:

pg_stat_get_backend_io() doesn't exist in 17, so we'll have to modify
the test to backpatch there.

Right. I think we can reset the shared I/O statistics and check pg_stat_io with
backend_type = 'client backend' (and the same filtering already used with
pg_stat_get_backend_io()).

This should be safe in the dedicated test cluster.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

#11Nazir Bilal Yavuz
byavuz81@gmail.com
In reply to: Bertrand Drouvot (#9)
Re: Fix ReadStream handling when heap rescan strategy changes

Hi,

On Wed, 18 Nov 2026 at 10:47, Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Hi,

On Tue, Nov 17, 2026 at 04:54:08PM -0500, Melanie Plageman wrote:

On Tue, Nov 3, 2026 at 1:40 AM Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

0001 takes a different approach (than [2]) and recreates the ReadStream when
initscan() changes whether a strategy is active. Ordinary rescans continue to
reset and reuse the existing stream. It uses the existing ReadStream APIs, which
could also make the backpatches easier.

Now that initscan() has to concern itself with the read stream because
it has to reset or end the stream before changing the strategy, then
change the strategy, and then make the new read stream, I think the
separation between initscan() and making the read stream is no longer
useful. In fact, the ping-ponging has become confusing. I think your
version might actually have read_stream_reset() getting called twice
in some cases since read_stream_end() also calls it. Attached v2 just
moves all the read stream setup into initscan() and adds some flags to
make it more clear when we want to rebuild the stream/strategy.

Thanks for looking at it!

Yeah, that makes sense to me. Having initscan() own the reset/end, strategy update,
and stream creation makes the ordering clearer (and also avoids the possible double
reset).

+1, v2 LGTM.

"
stream is allocated at the beginning of the scan and reset on rescan or
when the scan direction changes.
"

worth to change it to something like?

"
The read stream is allocated at the beginning of the scan. It is reset when
the scan direction changes. On rescan, it is reset if strategy use remains
unchanged, and recreated otherwise.
"

I think this is a nice improvement. This comment needs update after
this patch and I think what you recommended is correct and clear.

--
Regards,
Nazir Bilal Yavuz
Microsoft

#12Melanie Plageman
melanieplageman@gmail.com
In reply to: Nazir Bilal Yavuz (#11)
Re: Fix ReadStream handling when heap rescan strategy changes

On Wed, Nov 18, 2026 at 7:15 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:

On Wed, 18 Nov 2026 at 10:47, Bertrand Drouvot
<bertranddrouvot.pg@gmail.com> wrote:

Thanks for looking at it!

Yeah, that makes sense to me. Having initscan() own the reset/end, strategy update,
and stream creation makes the ordering clearer (and also avoids the possible double
reset).

+1, v2 LGTM.

Thanks for the reports and the review. I've pushed to master and back branches.

- Melanie