model-writtenrun 7324 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Jan 2020 - May 2021 · 21 entries
- Thomas Munro opened the thread with recovery-loop prefetching through PrefetchBuffer(), initially using a second WAL reader and POSIX_FADV_WILLNEED while anticipating eventual single-pass decoding and real asynchronous I/O #1.
- Tomas Vondra argued that storage concurrency was easier to configure than an arbitrary WAL-byte distance and recommended avoiding repeated prefetches; Thomas accepted tracking outstanding requests by their WAL positions #2 #3.
- The first indexed series implemented concurrency limits and reloadable settings, and the next revision added a monitoring view, counters and recovery logging #4 #5.
- Alvaro Herrera's prerequisite review led to consistent buffer-prefetch and WAL-receiver names and maintenance_io_concurrency as the main control, while discussion with Andres Freund led to an extensible result containing a recent buffer ID and an I/O flag #6 #7 #9.
- Andres's detailed review prompted nonblocking-read handling, statistics reset and persistence, clearer module boundaries and corrected queue accounting; the buffer-repinning demonstration remained explicitly experimental #10 #11.
- The buffer-manager prerequisites landed from #11 as 3985b600f5, and the consolidated receiver accessor patch from #13 landed as d140f2f3e2.
- Thomas deferred the main feature at the PostgreSQL 13 deadline because it needed more recovery testing, then resubmitted the remainder; Dmitry Dolgov's tests explored concurrency limits and showed that operating-system read-ahead could mask the benefit #14 #16 #17 #20.
- Tomas's later tests exposed early prefetch termination during archive recovery, and Thomas traced it to the second reader treating unavailable restored WAL as permanent end-of-input #23 #46 #49.
- Thomas replaced the two-reader design with a circular decoded-record buffer and XLogReadAhead(), eliminating duplicate decoding and introducing the recent-buffer shortcut together with recovery_prefetch and wal_decode_buffer_size controls #51.
- Testing the unified reader confirmed archive-recovery improvements but exposed stale configuration documentation and the unresolved problem that a blocking restore_command could stall replay while prefetching looked ahead #53 #54 #55 #58.
- Jakub Wartak supplied extensive throughput profiles and replicated-data checks, and Tomas demonstrated roughly fivefold faster catch-up with full-page images enabled; discussion then focused on avoiding redundant prefetches without adding excessive lookup overhead #64 #68 #70 #71.
- Andres recommended deferring further optimization and simplifying the atomic counters; Thomas adopted the simpler helper, separated ReadRecentBuffer() and accepted that the initial release could disable prefetching by default #66 #72 #80.
- Thomas integrated Horiguchi's callback-removal work, changing the reader to a caller-driven state machine and dropping access to written-but-unflushed WAL; Tomas questioned the lost overlap, but Thomas deferred it for lack of confidence before the deadline #83 #84 #85.
- ReadRecentBuffer() landed from #83 as 2f27f8c511, with the function rewritten to validate and lock the expected buffer before pinning it.
- The same series supplied the circular decoding buffer in f003d9f872 and the combined prefetch/recent-buffer implementation in 1d257577e0, with the feature disabled by default at landing #83 #85.
- Justin Pryzby's inline documentation and comment corrections became dc88460c24, after Thomas rewrote an unclear explanatory comment #87 #88.
- Noriyoshi Shinoda separately reported that the statistics-view documentation was embedded in the subscription section and supplied a corrective attachment, while Amit Kapila identified a missing statistics-file format bump #91 #92.
- Tom Lane's PowerPC recovery failures triggered renewed scrutiny of the combined landing, and Thomas proposed reverting and returning with stronger tests rather than stabilizing the late refactoring under release pressure #93 #96.
- Andres found a distinct infinite loop in XLogFindNextRecord() when a record crossed pages and supplied a found_start-state fix, then reproduced oversized allocation failures caused by trusting recycled WAL before validating its record header #109 #110.
- Tom's extended tests found occasional failures even before the new work, while Tomas could not reproduce them on x86 or Raspberry Pi systems; the independently confirmed reader bugs nevertheless remained grounds for withdrawal #113 #116 #118.
- Thomas reverted prefetching, circular decoding, their documentation changes and the callback-removal dependency in c2dc19342e, confirming the rollback in #122.
Nov - Dec 2021 · 2 entries
- After a six-month gap, Thomas returned with a redesigned pair of patches that restored callbacks, validated records before oversized allocation, separated block selection from I/O-depth control and removed the optional full-page-image setting, persistent averages, cached relation pointer and verbose logging #124.
- Tomas benchmarked the renewed design successfully, and Tom later reproduced the old PowerPC symptom on unpatched HEAD, removing that particular concern as a blocker #125 #143.
Mar - Apr 2022 · 9 entries
- The 2022 revisions moved recovery integration into xlogrecovery.c and added timeline barriers; Julien Rouhaud's reader review improved queue terminology and nonblocking-state handling, while Andres requested test coverage and proposed the portable try setting #147 #150 #151 #153 #155.
- Julien approved the final circular-buffer component, which landed from #155 with light cleanup as 3f1ce97346; Thomas then submitted the remaining prefetch patch alone #156 #158.
- Julien's prefetch review led to better documentation, statistics handling and naming, while Thomas fixed missing-relation checks, database filtering and repeated prefetches and selected try as the provisional default #160 #161.
- The remaining feature landed from #161 as 5dc0418fab, with further statistics-reset and computation changes, publication before waits and recovery completion, and final naming cleanup #162 #163.
- Justin again caught documentation inconsistent with the final default, producing dafae9707a; Thomas also removed obsolete wording about null statistics #164 #165.
- Shinoda resubmitted the statistics-section correction for the reintroduced feature, but Thomas explained that an independently reported fix had already been committed before he saw the attachment #166 #167.
- Simon Riggs questioned the benefit when full-page writes were enabled; Tomas and Thomas explained that evicted pages could still need prefetching and that the infrastructure would support later direct and asynchronous I/O work #169 #173 #175.
- Tom's pg_walinspect failure report led Thomas to identify a flush position ending inside a WAL record and move the fix to thread #45776; the matching attachment there, #506204, became ed57cac84d with light edits, so that commit belongs to the extension's thread rather than this diagnostic discussion #176 #178 #179.
- By the final message the PostgreSQL feature had landed, with real asynchronous I/O and other extensions left for future work; Thomas reported that Linux OpenZFS had also gained the required prefetch behavior in its development branch #175 #180.