model-writtenrun 9567 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Aug - Sep 2017 · 2 entries
- Tomas Vondra opened the thread with multivariate MCV lists and histograms left over after PostgreSQL 10 introduced the basic CREATE STATISTICS infrastructure, adapting that earlier work to the committed implementation #1.
- Adrien Nayrat found crashes when inspection functions received statistics of the wrong kind and supplied documentation corrections; Tomas fixed those problems and valgrind findings in the next full series #2 #4.
Nov 2017 - Apr 2018 · 7 entries
- After compatibility updates, Mark Dilger's testing exposed an unsafe assumption about array-analysis metadata; Tomas supplied a small serialization fix and then incorporated it with array tests and a simpler packed histogram representation #8 #10 #20.
- Mark's regression run exposed changed estimates, which Tomas traced to dependencies_clauselist_selectivity discarding information about already-estimated clauses; a new prerequisite preserved that information to prevent repeated estimation #22 #25.
- Documentation repairs and rebases kept the series usable through early 2018, while Mark confirmed successful testing and supplied the missing CREATE TABLE LIKE integration #27 #28 #30 #34 #38.
- Alvaro Herrera's review led to statistics carrying their own type OIDs, inspection functions accepting typed statistic values, removal of unproven match-count optimizations, and consolidation of the prerequisite into the MCV patch #36 #40.
- Dean Rasheed's tests then exposed clause-selection errors, boolean and NOT failures, and incorrect estimates outside the MCV list, prompting repairs and a separate experimental equality/ndistinct estimator #44 #47 #48 #49 #54.
- Dean questioned whether full-match tracking could handle general predicates, while Tomas retained top-level equality information and implemented a non-MCV estimator using remaining probability mass, ndistinct and separately estimated inequalities #55 #57 #59.
- The work missed PostgreSQL 11 because planner use of the statistics remained unresolved and review was incomplete; Dean and Tomas agreed it was not ready to commit #61 #63.
Jun - Sep 2018 · 4 entries
- Tomas restarted with a June rebase and adopted the single-column MCV-retention algorithm, while questioning how multivariate histograms should estimate partially matching buckets #64.
- Dean's subsequent testing demonstrated serious OR-clause overestimates and raised security checks and histogram-interpolation concerns; discussion of incremental corrections exposed further ordering and non-MCV complications #65 #67 #71 #75.
- Dean supplied a simpler replacement MCV estimator that stores each combination's independent base frequency and uses the difference from its observed frequency to correct ordinary selectivity, with substantially improved benchmark results #80.
- Tomas adopted Dean's base-frequency design and statistics-target sizing, while reworking histograms to condition on equalities and interpolate partial buckets; the justification for combining those interpolations remained uncertain #82.
Nov 2018 - Apr 2019 · 15 entries
- Following another gap, Tomas refreshed the series and fixed uninitialized-variable warnings reported by Thomas Munro #86 #88.
- Discussion of which combinations deserved MCV entries produced a relative-standard-error cutoff, adopted alongside Dean's documentation and code-review fixes; more elaborate ranking by departure from independence was deferred #89 #92 #98.
- Dean's type-change review simplified statistics invalidation, and David Rowley joined with extensive reviews of clause tracking, casts, serialization, allocation and documentation, producing further revisions #100 #102 #103 #106 #110.
- Alvaro recommended concentrating on one feature for PostgreSQL 12, and David, Tomas and Dean agreed to prioritize MCV lists while leaving the more uncertain histogram work for later #118 #119 #120 #121.
- Tomas's February revision accepted RelabelType nodes, repaired inspection-function output and removed obsolete MCV-building machinery while keeping the histogram patch rebased for future work #122.
- David supplied a cleanup patch, and Dean's March review clarified estimator responsibilities and advocated regression tests checking cardinalities rather than complete query plans #123 #124 #125 #126 #132.
- Bitmap-membership problems identified by Kyotaro Horiguchi and David were repaired, and the new statistics tests switched to a helper extracting estimated and actual row counts #133 #134 #138.
- The new tests exposed ALTER TABLE resetting relpages and reltuples, prompting an experimental preservation patch that was later removed after Dean recommended handling the pre-existing issue separately #141 #146 #150.
- Dean reproduced failures involving toasted values, leading to explicit detoasting, removal of an arbitrary 1MB storage limit, and subsequent exclusion of excessively wide values; the series also corrected the single-clause shortcut that bypassed multivariate estimation #145 #147 #150.
- The final full submission incorporated the attribute-array API cleanup and remaining review changes, with Tomas explicitly scheduling MCV lists for commitment and histograms for a later release #151 #153.
- The MCV component of #153 landed unchanged as 7300a69950, while Dean's final minimum-count and inconsistent item-limit findings were accidentally pushed as separate follow-up fixes, as Tomas explained in #159 after #155 and #156.
- Tomas resent the remaining histograms and added a separate patch converting the pre-existing ndistinct and dependency tests to the same cardinality-checking helper #159.
- John Naylor supplied a one-line comment correction in the landed MCV code, and Tomas confirmed that it was pushed #162 #163.
- At the thread's end, Michael Paquier recommended closing the long-running entry as committed; Tomas agreed to start a fresh thread for remaining work and, following Alvaro's recommendation, promised to push the test conversion #164 #165 #166 #167.
- The test-conversion component of #159 subsequently landed as dbb984128e, additionally removing two parallel-worker settings, while multivariate histograms remained deferred.