model-writtenrun 9551 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Oct 2018 - Jan 2019 · 18 entries
- Surafel Temesgen proposed COPY FROM WHEN in #1 to skip input rows that fail a Boolean condition without first loading an intermediate table.
- Corey Huinker advocated exposing COPY input as a queryable result set in #4 and #14, while Tomas Vondra argued in #18 that lightweight import filtering addressed a different use case from querying files through foreign data wrappers.
- Tomas Vondra rebased and reviewed the initial patch in #8, identifying unsupported-expression errors, silently ignored COPY TO conditions, missing tests, and a filtering goto that bypassed interruption checks and memory-context setup.
- Lim Myungkyu independently reproduced the COPY TO problem in #9, and Surafel Temesgen addressed the early review in #10 by rejecting unsupported uses, adding tests, and using continue to skip rows.
- Masahiko Sawada found collation failures while testing in #11, leading to collation assignment and expression preprocessing in #16, which Lim Myungkyu subsequently tested in #25.
- Daniel Verite investigated psql's parser and concluded in #22 that backslash-copy already passed through the new condition, so a separate client parser change was unnecessary.
- Adam Berlin argued in #23 that WHERE communicated row filtering more clearly than WHEN, but Tomas Vondra suggested FILTER in #24, which Surafel Temesgen initially adopted in #26.
- Tomas Vondra's review in #27 prompted clearer parser errors and negative tests in #30, while also exposing the question of whether filter functions should see rows inserted by the ongoing COPY.
- After Tomas Vondra proposed restricting volatile functions in #31, Surafel Temesgen temporarily prohibited them in #32; Alvaro Herrera cautioned against per-row command-counter increments and supported WHERE in #34.
- Robert Haas argued in #36 that volatile functions were needed for random sampling and that snapshot visibility should be documented, and Tomas Vondra accepted that reasoning in #37.
- Surafel Temesgen adopted WHERE and allowed volatile functions with single-row insertion in #38, then documented visibility limitations in #40 after Tomas Vondra's request in #39.
- Tomas Vondra landed the filtering feature as 31f3817402 from #40, adding documentation polish and a COPY TO rejection test, and announced the landing in #42.
- Andres Freund questioned expression memory lifetimes in #44, and the ensuing investigation found both an overbroad batching restriction in #49 and potentially unbounded expression-memory growth when nearly all subsequent rows were filtered out in #51.
- Tomas Vondra posted the batching-condition correction in #50, then combined it with separate per-tuple and per-batch contexts in #54, replacing alternating expression contexts with copying around batch resets.
- The volatility correction landed separately as 4a8283d0ec, extracting its unchanged hunk from #54, and Tomas Vondra resent the remaining memory fix in #56.
- The memory fix landed as 36a1281f86 from #56, with a further correction keeping input parsing and default-expression evaluation in the per-tuple context before forming tuples in the batch context.
- David Rowley measured an approximately four-percent regression from the extra tuple copying in #57, but opposed reverting the memory fix in #63 because it addressed the filtering leak.
- Andres Freund explained in #63 that his pluggable-storage work parsed input directly into reusable tuple slots, eliminating the batch context, and reported promising performance after correcting slot copying in #66.
Mar - Apr 2019 · 8 entries
- After a two-month gap, Andres Freund posted the tableam and slot-based COPY conversion in #67, bringing work from thread #38991 into this thread to address the performance concerns.
- David Rowley's review in #68 questioned retaining per-partition slot arrays while still flushing on every partition change, and he proposed globally limited buffering across partitions in #70.
- David Rowley implemented hash-based partition buffers in #72, reporting substantial speedups while acknowledging remaining correctness and cleanup work, then added error line tracking, per-buffer insertion state, and cleanup in #78.
- Andres Freund's reviews in #75, #77, and #80 led David Rowley to replace duplicate hash lookups with ResultRelInfo pointers and a linked list, use fixed-size arrays, and evict the oldest excess buffers in #82.
- Andres Freund found a remaining BulkInsertState leak in #83, which David Rowley fixed in #84; suggestions to increase the buffering limit or change array layout were left out of this version, as discussed in #84.
- Andres Freund committed the combined work of David Rowley, himself, and Haribabu Kommi as 86b85044e8 from #84, adding callback documentation, naming cleanup, combined flush loops, and a flush before non-batching partitions, as reported in #88.
- David Rowley inspected the landed code and supplied the obsolete-nbuffers cleanup in #89, which Andres Freund committed as 57a7a3adfe together with an assertion requiring the tableam multi_insert callback.
- The thread ends with Andres Freund confirming that final cleanup in #90, with the filtering feature, correctness fixes, and buffering rewrite all landed; broader CopyFrom decomposition and more general reuse of multi-insert remained future ideas raised in #67 and #73.