model-writtenrun 7840 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Jul 2020 · 1 entry
- Peter Geoghegan began with a prototype that reused deduplication infrastructure to delete obsolete duplicates from unique indexes immediately before a page split, targeting non-HOT update bloat without changing visibility rules #1.
Oct 2020 - Jan 2021 · 19 entries
- After a three-month gap, Peter proposed executor hints to recognize logically unchanged index entries and generalized the implementation to nonunique indexes, prioritizing heap blocks containing likely obsolete versions #2 #4.
- Early benchmarks showed stable index sizes and improved throughput, while Anastasia Lubennikova raised low-cardinality and replication concerns; Peter kept deletion subject to existing visibility limits and bounded speculative heap access #5 #6 #7.
- Peter expanded candidate collection to include posting-list and nonduplicate TIDs as inexpensive additional checks, temporarily incorporating Thomas Munro's sort-template work to accelerate sorting #10.
- Robert Haas questioned holding an index-page lock during heap I/O, and Simon Riggs requested stronger latency evidence; Peter retained the foreground design because cleanup was localized and bounded, with Haas accepting that rationale and Simon emphasizing longer tests #11 #12 #13 #17 #22.
- Granular deletion within posting lists improved low-cardinality indexes, but demanding benchmarks exposed CPU costs; Victor Yegorov independently reviewed and tested the patch and advocated abandoning unproductive heap searches sooner #21 #24 #25.
- Peter responded with bucketed block priorities, favorable contiguous blocks, byte-based space targets, and specialized shellsort, dropping the temporary sort-template dependency and substantially improving the difficult low-cardinality workload #29.
- The work became a four-patch series named bottom-up index deletion, with a proper tableam callback, a dedicated executor hint, and centralized selection among cleanup strategies #30.
- Victor's mixed-workload tests exposed UPDATE slowdowns and a bloom signature warning; Peter simplified premature deduplication and posting-list handling, and Victor's longer tests then found no visible throughput regression #31 #32 #33 #34 #36.
- Peter extracted and committed BTP_HAS_GARBAGE deprecation from #34 as cf2acaf4dc, with only comment edits and rebase adjustments after Victor's focused review, retaining flag maintenance for compatibility with older indexes #38 #41 #43.
- The remaining series gained heap prefetching, executor-state-based detection of unchanged index keys, user documentation, and an initially optional disable setting #45.
- Discussion of queue workloads led Peter to extend ordinary LP_DEAD deletion to check additional TIDs on heap blocks already being visited, rather than enabling speculative bottom-up deletion indiscriminately; the next series shared deletion machinery and made tableam support mandatory #46 #48 #51.
- Heikki Linnakangas challenged the two-array representation, specialized sorting, and block-count bucketing; Peter explained the locality benefits and subsequently unified the old conflict-horizon interface with the new deletion interface, leaving two patches and eliminating one sorting specialization #52 #53 #54.
- Documentation revisions and further review clarified the design, while Victor's queue tests with recurring five-minute snapshots showed greatly reduced index growth at similar throughput #57 #62 #63 #64.
- Peter removed the disable setting, adopted Victor's expression-walker name, consolidated Heikki's requested explanations, and added the incoming tuple's heap block as another simple-deletion opportunity in the final full submission #66 #67.
- Victor's final regression testing found an EXPLAIN-output mismatch caused by track_io_timing rather than the patch, and he confirmed successful testing after correcting that configuration #68 #69 #70.
- Peter posted experimental deltas that combined the arrays and removed shellsort, but measured a roughly two-to-three-percent throughput advantage for the existing implementation and explicitly declined to include those simplifications #71.
- Peter committed the executor hint from #67 unchanged as 9dc718bdf2 and its deletion patch as d168b66682, adding a small heap-block scoring adjustment, comment corrections, and the WAL magic bump before announcing both landings #72.
- After landing, Amit Kapila proposed remembering earlier unchanged-index hints on each page to trigger cleanup in more mixed workloads; Peter explained that earlier versions had used such a flag but that he preferred the conservative incoming-tuple trigger, while Andres Freund stressed that bitmap scans make reliance solely on LP_DEAD insufficient #75 #80 #84.
- Amit demonstrated a bulk-update slowdown when deletion could reclaim nothing; Peter ultimately acknowledged the measured effect and accepted the tradeoff without proposing a fix, while Amit deferred further conclusions pending better workload testing #86 #92 #93.
Nov 2024 · 2 entries
- The thread resumed in November 2024 with Andy Fan's questions about the landed design, and Peter distinguished the executor trigger from the snapshot-conflict horizon and explained when later scans can set LP_DEAD bits #95 #96 #99 #100.
- All three candidate changes remained landed with no revert or additional corrective patch in this thread; its final message leaves Andy's proposal to extend bottom-up deletion to additional DELETE-related opportunities unanswered #101.