model-writtenrun 5968 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Dec 2020 - Jan 2021 · 4 entries
- Andrey Borodin opened the thread with a reproducible missing-index-entry bug: CREATE INDEX CONCURRENTLY ignored prepared transactions, and his initial patch made the lock-wait machinery include them #1.
- Tom Lane found that the patch ignored VirtualXactLock()'s wait flag, while Michael Paquier questioned VXID validity checks and whether standby conflict handling needed different behavior; Andrey corrected the wait handling and added an isolation test #5 #8 #7.
- Noah Misch concluded that standby conflict resolution would not encounter these prepared-transaction locks, requested allocation fixes and a validity predicate that recognized prepared transactions, and corrected lock reference-count handling in the final submission #14 #17.
- The January repair landed unchanged from Noah's final submission as 8a54e12a38, crediting Andrey as author and Tom and Michael for earlier review #17.
May 2021 · 1 entry
- Andrey reported fresh corruption on an installation containing the fix, then supplied a July reproduction showing that a transaction could become prepared after CIC collected its VXID, escaping the intended wait #19 #20.
Jul - Dec 2021 · 25 entries
- Noah explained why permanent XIDs alone were insufficient: a transaction could open indexes before obtaining an XID, so the fix needed to follow its identity through PREPARE; Andrey responded by preserving VXIDs in prepared-transaction state and looking up their XIDs #23 #24.
- An attempt to resolve duplicate VXIDs by waiting for known prepared transactions first and selecting the oldest match was rejected because older matching transactions could involve unrelated relations; the next implementation waited for all matching prepared XIDs #27 #28 #30.
- Following Noah's testing proposal and Peter Geoghegan's endorsement of amcheck-based validation, Andrey added deterministic psql and probabilistic pgbench tests, but the stress test still found corruption #31 #34 #35.
- Debugging and a workload without prepared transactions demonstrated a second bug, independent of two-phase commit #36 #37.
- Noah traced that second bug to invalidations consumed while RelationBuildDesc() was constructing an entry that was not yet in the cache, and supplied a prototype that tracked in-progress builds and retried them after invalidation #38.
- Andrey integrated the relcache repair but still observed two-phase failures, eventually finding a preparation window where ProcArrayClearTransaction() cleared transaction identity before PostPrepare_Locks() transferred the locks #39 #45.
- Noah requested separate commits and tests for the ordinary-transaction and prepared-transaction bugs, along with longer timeouts and collision-free prepared-transaction names; Andrey split the tests and shared their pgbench infrastructure #47 #48.
- Further stress testing still failed, and Noah extended the relcache prototype to handle whole-cache invalidation; meanwhile Andres Freund challenged custom XID lists, allocation under a central lock, missing comments, and lookup overhead #50 #51 #53 #54.
- Andrey discovered that an XID could be visible in PGPROC before its transaction lock existed, removed the unsafe preference for ordinary XIDs, and added synchronization points to eliminate races in the deterministic test #56 #57.
- The next revision removed the custom allocated list in favor of lookup-and-retry waiting, added savepoint workloads, and documented preparation ordering; Andres's concern about overhead on systems with two-phase commit enabled remained under discussion #60 #61 #62.
- Noah supplied a restart test that exposed invalid recovered VXIDs, and Andrey fixed startup handling by assigning recovered prepared transactions an XID-based synthetic VXID #64 #65.
- A more extensive redesign to coordinate VXID and XID waits at a higher level was considered for performance, unnecessary snapshot waits, and possible starvation, but was not adopted because the benefits did not clearly justify its complexity #64 #66.
- Noah prepared two final patches, fixing a TwoPhaseStateLock leak, moving lock transfer instead of transaction clearing, dynamically growing the relcache tracking array, and handling errors and debug cache invalidation; Andrey tested them successfully #67 #68.
- The relcache half landed as fdd965d074 and the prepared-transaction half as 3cd9c3b921, both from #67, with final test changes for backpatchability, sleep removal, and stderr capture explained in #70.
- The new tests exposed buildfarm timeouts and HP-UX crashes, prompting separate investigations rather than an immediate revert of the index fixes #70 #74 #88.
- Tom Lane proposed combining the stress workloads into one pgbench run so index operations would remain concurrent with inserts, then used nonblocking advisory locks to avoid the deadlocks caused by blocking lock acquisition #77 #82.
- Noah verified the rewritten test's sensitivity, and Andrey extended it to prepared transactions and REINDEX CONCURRENTLY; Tom committed that revision with fewer iterations and removal of the unused background_pgbench() helper as 7f580aa5d8 #85 #87 #89.
- Tom's investigation implicated an old macOS kevent problem, while Noah distinguished DragonFly's failure to reap a completed child from the macOS backend's failure to notice incoming data #84 #96.
- Sandeep Thakkar reproduced the HP-UX crashes and supplied regression output and debugger information; Noah then supplied a separate TAP/GDB diagnostic script to investigate equalTupleDescs() #103 #110 #111.
- Semab Tariq continued the HP-UX testing while Noah revised the diagnostic script to locate the executable, avoid premature exits, collect more state, and step past the function prologue #115 #116 #118 #119 #121.
- The resulting disassembly exposed an invalid branch destination, and comparison with another build confirmed a toolchain bug rather than a regression requiring reversal of the CIC fixes #123 #127.
- Andrey proposed a PostgreSQL 11 adaptation of the single-pgbench tests using client_id instead of advisory-lock result handling, but Noah preferred a solution that also repaired the affected recovery test; the attachment was not shown landing in this thread #132 #133.
- Thomas Munro reproduced the remaining DragonFly hang and reduced it to a small C program demonstrating incorrect select() behavior on a pipe, then filed an operating-system bug report #137 #138 #141.
- Tom supported the report with the readiness requirements and successful tests on other operating systems, and the participants chose to await an upstream repair instead of immediately disabling the PostgreSQL tests #142 #143 #144.
- DragonFly supplied a fix, the buildfarm host was subsequently upgraded, and Mikael Kjellström reported that all branches passed their first run; the database fixes remained committed, with further buildfarm passes still needed to confirm the environmental repair #145 #146 #147.