model-writtenrun 3138 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Mar 2021 · 6 entries
- Robins Tharakan opened the thread with a working transaction-ID-limit override for upgrades involving hundreds of millions of large objects, reporting a successful but expensive upgrade of 500 million objects #1.
- Magnus Hagander traced the consumption to separate restore transactions for object creation and ownership changes, while Robins reported that a single transaction instead exhausted locks; this redirected discussion toward batching #6 #7.
- Jan Wieck demonstrated that a single restore transaction would require impractical lock memory, and Tom Lane proposed grouping large objects by owner and ACL to reduce client-side TOC memory #12 #14.
- A database-ownership detour produced two alternative fixes and Tom's diagnostic test, but Jan withdrew the upstream bug claim after reproducing correct behavior in a pristine checkout #19 #23 #27 #28.
- Jan's replacement implementation combined each large object's metadata commands, moved them into the parallel data phase during binary upgrade, and batched transactions, reducing his pathological test from fifteen hours to roughly ninety minutes #30.
- Zhihong Yu's review removed redundant transaction state, while Bruce Momjian's suggestion led to generic dump/restore option forwarding after discussion of shell quoting and interface consistency #31 #33 #34 #47.
Dec 2021 · 1 entry
- After a quiet period, Justin Pryzby noted that separate work had eased the original XID limitation but that memory problems remained; Nathan Bossart subsequently revived and rebased Jan's patch #48 #49.
Aug - Oct 2022 · 1 entry
- Jacob Champion objected that arbitrary dump and restore options could compromise upgrade correctness, and Sachin Kumar later resumed the work with a rebase and a fix for transaction output being emitted after file closure #50 #52 #55 #58.
Nov 2023 - Jan 2024 · 6 entries
- Tom Lane and Andres Freund favored automatic behavior over additional controls, leading Sachin to replace option forwarding with large-object counts and heuristic restore scheduling #56 #57 #59.
- Tom then replaced the blob-specific approach with general --transaction-size batching, retaining commits between parallel worker tasks because dependent tasks must immediately see their results #61.
- Tom expanded that work into grouped large-object metadata and data entries, explicit dependencies, parallel restoration, and an archive-format change, while leaving batch-size controls and consolidation of repeated ACLs open #62.
- Robins tested the four-patch series across object counts, object sizes, and worker counts, reporting substantial dump and restore improvements #65 #67.
- Sachin supplied an additional section-based parallel pg_restore patch, but Tom rejected it because of restore-order risks and the loss of useful cross-database parallelism; that proposal was excluded from subsequent versions #68 #69 #70 #76.
- Nathan reviewed the new design and suggested named constants, which Tom incorporated before adding shared ACL storage and reducing each concurrent upgrade job's transaction size to keep aggregate lock use bounded #71 #76 #77.
Mar - Apr 2024 · 3 entries
- Tom deliberately left large-object comments and security labels ungrouped because he expected them to be uncommon, and deferred user controls pending field experience #77.
- Laurenz Albe's final-series tests demonstrated substantial memory and disk savings, Michael Banck reported promising earlier pg_dump benchmarks, and Tom ruled out backpatching the archive-format change #78 #80 #81 #84.
- After Nathan's final review, Tom retained the fixed transaction-size policy and committed the six-patch submission as two exact combinations: large-object restructuring in a45c78e328 and transaction batching in 959b38d770 #77 #86 #87 #88.
Jul 2024 · 6 entries
- Justin subsequently reported backend out-of-memory failures during upgrades of wide partitioned tables, bisected the regression to transaction batching, and supplied a compact reproducer showing gigabytes of cache growth #89 #92.
- Tom traced the growth to repeated pg_attribute updates invalidating a newly created table's relcache entry and posted a draft combining inherited-column updates into one command #94.
- Alexander Korotkov independently confirmed the diagnosis and suggested counting semicolons as an inexpensive upper-bound estimate of SQL commands; Tom initially preferred reducing emitted commands but retained the counting idea for difficult cases #95 #96 #97 #98.
- Tom's completed fix consolidated updates for dropped columns, inherited columns, and inherited constraints, then added adaptive transaction accounting because inherited constraints could still cause excessive memory use; he declined to merge all ALTER TABLE commands because that would undermine the accounting and introduce additional backend risks #99.
- The first two fixes from that submission landed together without code changes in 2fa989e6a3, and its semicolon-counting patch landed unchanged in 81db073a28, with both fixes backported to PostgreSQL 17 #99.
- Alexander's explicit approval of the completed July series arrived after those commits, so it does not establish pre-commit review of the final patches #100.
Mar 2026 · 1 entry
- At the thread's last message, the four candidate changes had landed, but Nikhil, posting as PP L, revived the partition case with an inline proposal to omit redundant attislocal updates entirely and reported roughly 33–36% faster upgrades; no response or landing for that new proposal is present #101.