model-writtenrun 5628 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
- David Rowley proposed replacing tuplesort's tuple-storage AllocSet with a Generation context, reporting lower memory consumption, fewer disk spills, and substantial benchmark improvements from a tiny initial patch #1.
- Andres Freund argued that a fixed large allocation would waste memory on small sorts and recommended growing blocks and a keeper block; he also raised expanded-datum allocation concerns, later suggesting those could be addressed separately without excluding datum sorts #3 #9.
- Tomas Vondra supplied separate allocator patches for growing blocks, retaining an empty block, and computing an oversized-chunk threshold #5; David questioned the keeper semantics, while Tomas explained that his block was intended as a reuse cache and added benchmarks while correcting a faulty allocChunkLimit initialization #6 #8.
- David's benchmarks showed that the allocator refinements did not initially preserve all the original speedup, and subsequent investigation considered faster block growth, its memory-waste tradeoff, a keeper-accounting assertion failure, and using planner estimates to choose allocation sizes #10 #13 #15 #16 #17.
- Tomas's September benchmark revision exposed large regressions that depended on prior allocations within the backend, with page faults appearing in the slow cases #18.
- Ronan Dunklau reproduced those regressions in December and used added Generation probes to trace them to glibc's dynamically adjusted mmap and trimming thresholds, showing that malloc tuning could substantially change both baseline and patched performance #19 #21 #22.
- Tomas encouraged investigating malloc tuning while raising portability and memory-reuse questions, and Ronan proposed pursuing that work separately #23 #24 #26.
- David revived the tuplesort work with planner-derived estimates for tuple storage and the tuple array, including more conservative sizing when the sort would exceed work_mem, but described the implementation as unfinished #28.
- Ronan then posted configurable glibc tuning and a measurement-only malloc_stats extension; Tomas questioned compile-time allocator detection, and Julien Rouhaud reported Windows build failures in the diagnostic extension and confusion about the thread's scope #29 #30 #32.
- David clarified that growing Generation blocks remained a prerequisite because many tuplesort callers lack useful planner estimates, while Ronan's jemalloc comparison showed that the optimization still helped there but that much of the larger glibc improvement depended on allocator behavior #33 #35.
- Andy Fan reviewed the growing-block change and benchmarked it with the original tuplesort substitution, recommending that this smaller useful combination proceed before planner-estimate and further allocator experiments #36.
- David took over Tomas's allocator patches, combined the keeper with the context allocation, and deferred malloc tuning to separate work; testing also exposed a bounded-sort regression from repeated allocation and freeing, which he initially addressed by retaining the empty current block #39.
- Andres suggested compacting fragmented tuple storage, but Tomas questioned how existing pointers would survive, and David concluded that such compaction would need cooperation from the allocator's caller #40 #41 #42.
- David replaced current-block retention with a single reusable freeblock, preserving a separate keeper block, and increasingly favored keeping bounded sorts on AllocSet #42.
- The final design reused cached blocks only when completely empty to preserve FIFO allocation locality, introduced extensible tuplesort option flags, and selected AllocSet whenever TUPLESORT_ALLOWBOUNDED was requested #43.
- Justin Pryzby found an overlooked DTrace argument and reposted the complete final series with a one-line correction #44.
- David committed the allocator component from that repost as 1b0d9aa4f7, with formatting and comment cleanup, retaining the growing blocks, keeper, and freeblock implementation developed with Tomas #44.
- David committed the option-flags component together with Justin's DTrace correction as 77bae396df, with comment wrapping relative to the submitted code #44.
- David committed conditional Generation use for unbounded sorts as 40af10b571, with only indentation changes to the submitted implementation, and announced that all three patches had landed #44 #45.
- Afterward, Noah Misch observed that existing boolean arguments remained compatible with the random-access flag and found no tuplesort_set_bound callers among the extensions he checked; Peter Geoghegan and David accepted the practical compatibility situation, with David explaining the fragmentation risk for callers that omitted the bounded-sort flag #46 #47 #48.
- The planner-estimate experiment and diagnostic extensions were absent from the landed series, while glibc tuning remained separate unfinished work: in June 2023 Tomas asked whether it still helped after the allocator improvements, and Ronan confirmed ongoing evaluation and promised a new thread #28 #29 #49 #50.