Use streaming read I/O when enabling data checksums online
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t139827psql -h localhost -U postgresBuilt from patchset v5 (message #5), August 23, 2026 at 06:10 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t139827_5 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t139827_5 && git checkout t139827_5Patchset v5 (message #5) is on t139827_5
Hi hackers,
Attach a simple patch for $subject. I think we prefer streaming IO instead of
ReadBufferExtended(). Do I miss something?
--
Regards,
ChangAo Chen
On 3 Jun 2026, at 13:10, cca5507 <cca5507@qq.com> wrote:
Attach a simple patch for $subject. I think we prefer streaming IO instead of
ReadBufferExtended(). Do I miss something?
Thanks! This code was written well before read streams existed, and recent
work was focused on correctness of operation, that's the main reason it's using
ReadBufferExtended still. We probably want do something like this once the
tree opens up after the v19 feature freeze.
--
Daniel Gustafsson
Attach a simple patch for $subject. I think we prefer streaming IO instead of
ReadBufferExtended(). Do I miss something?Thanks! This code was written well before read streams existed, and recent
work was focused on correctness of operation, that's the main reason it's using
ReadBufferExtended still. We probably want do something like this once the
tree opens up after the v19 feature freeze.
Get it! I created a CF entry for this:
https://commitfest.postgresql.org/patch/6841/
--
Regards,
ChangAo Chen
The following review has been posted through the commitfest application:
make installcheck-world: tested, passed
Implements feature: tested, passed
Spec compliant: not tested
Documentation: tested, passed
Hi,
Contents & Purpose
==================
This patch converts ProcessSingleRelationFork() in
src/backend/postmaster/datachecksum_state.c from a plain per-block
ReadBufferExtended() loop to the new ReadStream API.
This is a pure internals/performance change: the external contract is
unchanged.
No documentation changes are included, which seems right since no
user-visible behavior changes.
Initial Run
===========
The patch applies cleanly to HEAD and builds without warnings.
I built it and ran it end-to-end by hand: initialized a cluster
without checksums, created a ~200,000-row/5,883-page table, and called
pg_enable_data_checksums(). Progress was visible via
pg_stat_progress_data_checksums while the new stream-based scan was
running, and data_checksums correctly flipped to "on" on completion.
I then independently verified correctness with the offline pg_checksums
--check tool: 9,434 blocks scanned across the cluster, 0 bad checksums.
I also exercised the abort path (the "success = false; break;" case
when checksums are disabled while an enable is in progress): I re-ran
pg_enable_data_checksums() with a cost_delay/cost_limit slow enough to
catch it mid-scan, then called pg_disable_data_checksums()
concurrently. The worker logged "data checksums processing was aborted"
and cleanly transitioned to disabled, and the server stayed healthy
afterward.
Performance
===========
I benchmarked this against pre-patch HEAD using a ~7.5 GiB table
(956,815 blocks) with debug_io_direct=data to force real synchronous
reads, 5 repetitions per configuration, median reported:
- io_method=sync:
HEAD = 460003ms, 958482 reads, 299082ms io_time
patch = 383069ms, 60248 reads, 264387ms io_time
- io_method=worker:
HEAD = 458885ms, 958482 reads
patch = 352155ms, 60248 reads
There's a ~16x drop in read syscalls (matching the default
io_combine_limit). My test storage is SSD, so the clock time multiplier
may understate the win on higher-latency disks.
Nitpicking & Conclusion
=======================
One nit: the new "BlockRangeReadStreamPrivate p;" declaration is not
properly indented.
Please run pgindent and then I'll be +1 for committer review.
Best regards,
Enrique.
One nit: the new "BlockRangeReadStreamPrivate p;" declaration is not
properly indented.Please run pgindent and then I'll be +1 for committer review.
Fixed, thanks!
--
Regards,
ChangAo Chen
Hi ChangAo, for some reason the commit fest entry and the thread in
psql-hackers is not showing your last email with the fixed patch.
I've changed the status to "Ready for Committer" anyway and hope that it'll
be shown soon.
Best regards,
Enrique.
El dom, 12 jul 2026 a las 18:06, cca5507 (<cca5507@qq.com>) escribió:
Show quoted text
One nit: the new "BlockRangeReadStreamPrivate p;" declaration is not
properly indented.Please run pgindent and then I'll be +1 for committer review.
Fixed, thanks!
--
Regards,
ChangAo Chen
Import Notes
Reply to msg id not found: tencent_E94FF8A0DEAE2AD4B750BC3626654D261B05@qq.com
On 12 Jul 2026, at 18:06, cca5507 <cca5507@qq.com> wrote:
One nit: the new "BlockRangeReadStreamPrivate p;" declaration is not
properly indented.Please run pgindent and then I'll be +1 for committer review.
Fixed, thanks!
Thanks for this work, I have it on my radar to review and commit for v20, but
will wait for the work of cleaning up and fixing the online checksums code
which is currrently ongoing for the beta period.
--
Daniel Gustafsson