model-writtenrun 7568 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Sep 2017 · 2 entries
- Ildus Kurbangaliev proposed extensible per-datum TOAST compression based on unpublished work by Nikita Glukhov, with handler callbacks, per-column options, catalog records, and a custom compressed-value flag #1.
- The first revisions added pg_dump and a compression-options cache, removed type-wide compression configuration, and repaired pg_upgrade support #2 #5 #12.
Nov 2017 - Apr 2018 · 6 entries
- Robert Haas warned that dropping a compression implementation could leave unreadable stored values, while Tom Lane proposed dependencies between columns or types and their compression methods #8 #10.
- Tomas Vondra’s extension tests exposed a broken cache key, copied-value failures, and compression-flag errors; Ildus added recompression checks, and Tomas confirmed fixes with real-data benchmarks #16 #26 #30 #35 #36 #37.
- Alvaro Herrera proposed CREATE ACCESS METHOD for compression, while Robert and Ildus developed PRESERVE lists so older formats could remain permitted without a rewrite; the next redesign adopted both ideas, builtin pglz, and partition support #41 #63 #64 #67 #90.
- Ildar Musin’s reviews led to INCLUDING COMPRESSION for CREATE TABLE LIKE, consistent storage-setting rules, cache and macro fixes, and stronger documentation and tests #91 #93 #95 #97.
- After several rebases, Ildus removed the untested cmdrop callback rather than retain an unverified extension hook #103.
- Konstantin Knizhnik proposed broader WAL and client-traffic compression, but Alexander Korotkov and Ildus kept those outside this series because recovery cannot depend on catalog-defined handlers and transport compression needs a separate design #104 #105 #106 #108 #110.
Jun - Jul 2018 · 1 entry
- Ildus split the work into eight patches and added zlib with dictionary support as a concrete extensibility example, then improved its tests, documentation, and upgrade handling #112 #114 #116.
Feb - Mar 2019 · 2 entries
- Repeated rebases through 2019 did not resolve the high-level design concerns, and Tomas observed that placing documentation and tests at the end prevented independently committing earlier pieces #122 #128.
- Tomas supplied a separate jsonbd extension compatibility patch for experiments, but a detached-worker failure remained; Ildus subsequently added slice-decompression support before the main proposal stalled again #132 #138 #139.
Jun 2020 - Jan 2021 · 8 entries
- Robert revived the work in 2020 around smaller independently useful changes, and Andres Freund recommended compact builtin identifiers in unused size bits; Dilip Kumar then took over and posted a three-part rebased series #140 #141 #145 #148.
- Mark Dilger’s pglz compatibility review led Dilip to preserve the legacy format and use two spare size bits for builtin methods, producing a pglz/zlib implementation with ALTER support #152 #153 #155.
- Tomas rejected the bulky fixed PRESERVE representation and urged catalog and dependency tracking, leading Dilip to introduce pg_compression and maintain preserved methods through pg_depend #158 #163 #167 #170 #172.
- Dilip restored custom methods, moved faster LZ4 into the builtin set and zlib into an extension, and added PRESERVE ALL and compression options; dictionary-dependent options remained unsupported because only the current method’s options were retained #175 #176 #181 #182 #183.
- Robert’s detailed review requested cheaper handlers, stronger tests, and pg_column_compression, while Tom and Alvaro supported using pg_am; Dilip incorporated those changes into another architectural revision #188 #194 #196 #201.
- Andrey Borodin explored an additional LZ4 WAL full-page-image patch and measured reduced compression overhead, but that side series remained outside the eventual TOAST landing #204 #206 #207.
- Justin Pryzby’s upgrade testing exposed missing compression metadata and preserved dependencies, which Dilip repaired, while Neha Sharma supplied detailed test-coverage results #208 #211 #216 #217 #220.
- Justin supplied default_toast_compression and psql output controls, and packaged review fixups with CI-only switches that enabled LZ4 for testing rather than for commitment #223 #249.
Feb 2021 · 2 entries
- Robert demonstrated that compressed values inside composite records escaped dependency tracking; Dilip’s response evolved from a narrow fix into container decompression and audited raw-Datum APIs, with expanded-record benchmarks exposing performance costs #244 #245 #246 #252 #254 #262 #263 #276.
- Justin found older-LZ4 slicing failures and GUC-cache problems, leading to a full-decompression fallback and simpler GUC handling #267 #266 #272.
Mar 2021 · 20 entries
- Dilip and Robert ultimately abandoned runtime-added compression methods because composite values and WAL could outlive their handlers; the builtin-only rewrite removed compression catalogs, PRESERVE, and forced ALTER rewrites #277.
- The narrowed series consolidated its compression code, while Justin worked through macOS pkg-config failures and Dilip explored recompression during INSERT SELECT, CTAS, VACUUM FULL, and CLUSTER #294 #297 #301 #302 #310 #317.
- Robert and Andres challenged the cost of recompressing ordinary inserts, and Dilip’s measurements confirmed substantial overhead; the next design preserved incoming compressed values, stored method bits in external pointers, and recompressed only during VACUUM FULL or CLUSTER #318 #319 #323 #330.
- Final review simplified dispatch and LZ4 slice fetching, and Robert requested merging the GUC and ALTER pieces into the core feature; Dilip supplied that combined version with a separate recursive-ALTER experiment #337 #339 #340.
- Robert committed the core component of that series with light editorial and documentation changes as configurable LZ4 TOAST compression; the preparatory patches, test-only build switch, and recursive-ALTER experiment were excluded #340 #345 bbe0a81db6.
- Immediate review identified dump-portability problems, prompting Robert’s default-suppression proposal and competing implementations from Justin and Dilip; testing showed Dilip’s version lost the setting in non-text archives #347 #353 #368 #369 #377.
- Tom committed Justin’s archive-state and separate ALTER-command approach with a server-version guard and cleanup, making compression clauses less disruptive to portable restores #368 #380 aa25d1089a.
- Tomas discovered that BRIN summaries can have a different type from the indexed attribute and supplied a fallback to default compression; Justin’s branch and comment corrections were incorporated before landing #371 #375 #376 #391 882b2cdc08.
- Testing with a non-pglz installation default exposed unstable regression results, and Tom committed Dilip’s explicit pglz test setup with an explanatory comment #393 #395 fd1ac9a548.
- Robert proposed removing pkg-config, but Tom instead retained it, separated compiler and linker flags, and restored an explicit library probe; this replaced Justin’s interim configure fix #354 #392 #400 4d399a6fbe.
- Tom then found garbage bytes in oversized substring requests with LZ4 and posted a guard; Dilip generalized it to the shared decompression path rather than only the LZ4 routine #397 #404 #406 #413.
- Justin’s remaining cleanup series produced separate commits correcting char-versus-OID handling and removing an unused macro, documenting the GUC and dump option with revised wording, and rejecting invalid compression names in CREATE and ALTER #405 #408 #411 226e2be387 24f0e395ac a4d5284a10.
- Tom removed the unsupported lz4/lz4.h configure probe after observing that the C code had no matching include support #401 2c75f8a612.
- The slice discussion also led Tom to clarify public TOAST macro names, distinguish payload size from total datum size, fix an ignored macro argument, and make compression-bit shifts unsigned #409 #414 #415 aeb1631ed2.
- Tom landed Dilip’s shared oversized-slice guard after adjusting comments, accessor naming, and the comparison cast, avoiding both excessive allocation and LZ4 partial-decompression misbehavior #413 #414 063dd37ebc.
- Robert substantially rewrote Justin’s enum-GUC proposal to use catalog char values and reduce header dependencies; reverse review refined declarations and documentation, and the final attached version landed unchanged #419 #420 #421 #424 #440 e5595de03e.
- Jaime Casanova reported an expression-index insertion assertion, and Dilip’s fix grew regression coverage and proposals for index display and independent ALTER INDEX compression #427 #429 #430 #432 #433.
- Robert clarified that ordinary index columns already follow table settings and expression columns must use the runtime default; Dilip dropped independent ALTER INDEX support, and Robert committed only the bugfix with comment edits, leaving the display patch out #434 #438 #444 #448 #450 5db1fd7823.
- Tom argued that dumps of pre-v14 servers should explicitly preserve pglz behavior, while Robert preferred letting the destination default apply; after discussion, Tom implemented source-behavior preservation unless --no-toast-compression was requested #436 #439 #442 #447 #454 54bb91c30e.
- The delivered feature therefore remained builtin pglz/LZ4 TOAST compression with follow-up fixes, while runtime custom methods, dictionary options, PRESERVE, and the separate WAL-compression experiment did not land through this thread’s candidate commits #277 #345 #207.
Apr 2021 · 1 entry
- The core commitfest entry was marked committed, and the thread ended with Robert’s storage-documentation update and Justin’s missing-word correction agreed; that final patch landed five days after the last message #455 #456 #457 #458 e8c435a824.