model-writtenrun 8197 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Apr 2020 · 2 entries
- Jeff Davis opened the thread to decide whether PostgreSQL 13 should enable disk-based HashAgg planning by default, initially favoring enabling it during beta to gather experience #1.
- Justin Pryzby and Robert Haas questioned a setting whose disabled state still allowed execution-time spills, while Jeff explained that the settings served upgrade compatibility and grouping-set test coverage #2 #6 #7.
Jun 2020 · 7 entries
- Melanie Plageman demonstrated that altered table statistics could exercise grouping-set spills without a dedicated setting, and Justin proposed clarifying the remaining setting's planner-only meaning #8 #10 #9.
- Jeff implemented those proposals in 92c58fd948, removing enable_groupingsets_hash_disk, revising regression tests, and renaming enable_hashagg_disk to hashagg_avoid_disk_plan with inverted polarity; the default-policy question remained open #15.
- Justin then reported that the renamed documentation entry was out of alphabetical order, prompting its relocation in 7ce4615601 without an attached patch #18.
- The discussion distinguished changed planner choices from execution-time spilling, with David Rowley proposing additional enable_hashagg modes and Tom Lane resisting an overloaded planner-and-executor switch #26 #33.
- Andres Freund supplied examples showing substantial spilling regressions and explained that raising work_mem could also increase memory consumption by unrelated operations, motivating a separate hash-memory allowance #36 #60 #63.
- Robert Haas and Andres considered extra execution-time memory, but Jeff warned that granting it only after underestimation could penalize improved statistics and multiply memory overruns across concurrent queries #38 #40 #47 #50.
- Peter Geoghegan narrowed the immediate solution to a hash-memory allowance used consistently by planning and execution, and Tomas Vondra accepted that direction while noting that a finite limit would not reproduce PostgreSQL 12's unlimited behavior #83 #89.
Jul 2020 · 13 entries
- Peter's first attachment implemented an independent hash_mem setting defaulting to 16MB, while already proposing a multiplier to preserve existing work_mem tuning; Justin's review also identified the danger of an independent default remaining unchanged when work_mem increased #92 #99.
- After Tom reviewed the initial patch and favored a multiplier of at least 1.0, Jeff paused his planned unlimited-memory escape hatch; discussion of the multiplier's range also established the need to clamp the calculated memory limit #134 #135 #137 #140.
- Tomas objected to increasing everyone's memory allowance by default, and Peter accepted Tom's proposal to start hash_mem_multiplier at 1.0, preserving existing behavior #161 #162 #163.
- A side discussion identified that major upgrades could lose ALTER SYSTEM settings, and Daniel Gustafsson confirmed that pg_upgrade documentation omitted postgresql.auto.conf and should also address included configuration files #187 #189.
- Peter posted the multiplier implementation with a shared get_hash_mem() helper and a preparatory variable rename; Jeff reviewed it, and Peter reported that the rename had already landed #192 #193 #194.
- Jeff initially wanted to retain hashagg_avoid_disk_plan for one release, but Tom argued that compatibility settings become difficult to remove, leading Jeff toward improving costing instead; broader query-level memory management remained a longer-term research project #195 #196 #198 #201.
- Peter's benchmarks showed that hash aggregation could use less memory than sorting yet suffer more sharply when memory was reduced, and his next series added removal of hashagg_avoid_disk_plan while simplifying the multiplier's planner changes #210 #211.
- Tomas's fresh tests questioned the presumed random-I/O disadvantage but exposed excessive spilling and anomalous batch counts; Jeff traced overpartitioning to using tuple counts as group estimates during recursive spilling #212 #214 #225.
- Jeff supplied an independent HyperLogLog patch to estimate spilled-group cardinality, and Peter's testing confirmed smoother partition counts and a reduction from 2,424 batches to 20 in one case #230 #233 #234.
- Tomas also resolved an apparent accounting anomaly: growing aggregate transition states could exceed work_mem, and EXPLAIN was correctly reporting that excess #235 #236.
- Álvaro Herrera's review moved the get_hash_mem() declaration into miscadmin.h in the final series, after which Álvaro and Jeff checked the planner cleanup; its removal component landed with only comment edits as bcbf9446a2 #237 #240 #241 #243 #244.
- Jeff committed the HyperLogLog attachment unchanged apart from rebasing as 9878b643f3, while identifying remaining estimation overhead for separate optimization #245.
- Peter committed the final series' multiplier component with additional documentation and comment edits as d6c08e29e7, considering the original problem addressed while leaving a true compatibility escape hatch contingent on future evidence #240 #246.
Aug 2020 · 1 entry
- Bruce Momjian later supplied the pg_upgrade documentation patch, which landed unchanged as 50ed605b3e; his final message confirmed application, leaving no attached work pending here, with costing and HyperLogLog overhead treated as separate follow-ups #247 #249 #243 #245.