model-writtenrun 7745 - may be wrong, links go to the source messages. Commit-by-commit LLM analysis is in the Commits dialog.
Jun - Jul 2020 · 3 entries
- Tomas Vondra opened the thread with a postgres_fdw-only batching prototype that reduced his million-row insert benchmark from roughly 64 seconds to 4.6 seconds, while identifying unresolved row-count, RETURNING, configuration, and consistency questions #1.
- Ashutosh Bapat favored server/table configuration and discussed remote statement semantics, Etsuro Fujita favored COPY for bulk loading, and Tomas explained why accurate row counts and INSERT SELECT still justified an explicit batching API #3 #4 #8.
- Andres Freund proposed shared DML infrastructure and pipelined execution, while Andrei Lepikhov questioned duplicating COPY's buffering machinery; these broader changes were discussed but did not become prerequisites for this patch #10 #11 #12.
Oct 2020 - Feb 2021 · 21 entries
- After several months, Takayuki Tsunakawa offered to take over implementation, and Tomas welcomed the handoff while highlighting remote triggers that can skip inserted rows #14 #15.
- Takayuki introduced executor-managed batches and a bulk-insert FDW callback, reporting an elevenfold improvement, then moved buffering into per-relation state and added a configurable batch limit following Tomas's review #17 #18 #23.
- Tim Colles raised the changed frequency of remote statement triggers, and Takayuki confirmed that batching reduces those firings and noted gaps in existing trigger documentation #21 #22.
- Tomas supplied a substantial alternative using separately prepared single-row and full-batch statements, server/table options, and list constructors, also exposing a missing partition helper in the preceding submission #24.
- Takayuki argued for rebuilding statements to match variable batch lengths, and Tomas accepted that approach provided deparsing stayed in deparse.c and the FDW exposed its chosen batch size through a callback; the next patch integrated those decisions and regression coverage #25 #26 #27.
- Further review standardized batch terminology, cached the batch size once per relation, and added EXPLAIN reporting instead of repeatedly consulting the FDW for every tuple #28 #30 #32.
- Craig Ringer advocated an asynchronous interface that could overlap batches, but Tomas judged the necessary executor redesign too broad for this feature, and Craig agreed to leave it separate #31 #39 #41 #42.
- The implementation retained INSERT rather than substituting COPY because of their different semantics and use cases, and Tomas resumed the patch in January with a rebase and small cleanups #46 #47.
- Amit Langote's testing exposed an AFTER ROW trigger crash, leading Tomas first to add a guard and then to move RETURNING and trigger eligibility decisions into the FDW's batch-size callback #49 #50 #51.
- Amit's subsequent review changed callback naming and partition flushing, then moved batch-size initialization into ExecInitModifyTable and ExecInitRoutingInfo so it occurred once during initialization #53 #55 #56 #58 #60 #61.
- Tomas fixed EXPLAIN without ANALYZE and added tests, and Takayuki consolidated the series around a shared option lookup with batching disabled by default #63 #64 #67.
- Amit questioned initialization outside the result-relation loop, but Takayuki explained its dependency on completed RETURNING setup; partition batch-size reporting in EXPLAIN also remained a separate improvement #68 #69 #71 #72.
- Tomas committed the consolidated patch with only comment polishing as b663a41363, adding the batching API and postgres_fdw implementation #67 #73.
- Buildfarm failures immediately exposed an out-of-bounds resultRelInfo pointer after junk-filter initialization, and Tomas posted a relocation fix while Zhihong Yu proposed accessing the array's starting pointer directly #74 #76 #77.
- Tom Lane required initialization of every result relation, Amit recommended an INSERT-only outer guard, and Takayuki removed a redundant inner check; Tomas committed the resulting lightly edited fix as 920f853dc9 #83 #86 #90 #92 #94 #96.
- Ian Lawrence Barwick supplied a one-line correction to the documented ExecForeignBatchInsert planSlots pointer type, which later landed unchanged as c15283ff42 #97.
- Amit found that cross-partition UPDATE could invoke batching initialization with update state whose batch size was zero, and proposed restricting tuple-routing initialization to plain INSERT operations #98.
- Tomas requested a regression test, Amit added a simplified local-join reproducer, and Takayuki reviewed the state-pointer assignment; subsequent revisions refreshed the patch and clarified its explanation #100 #101 #102 #105 #108 #109.
- Tomas questioned whether cross-partition inserts could support batching, prompting Amit to post a larger alternative that selected auxiliary insert state and flushed all routed partitions #110 #111.
- Tomas deferred that broader implementation to another commitfest and committed the smaller restrictive version unchanged as 927f453a94, with a rewritten explanation of the core/FDW mismatch #109 #112 #113.
- The thread ended with the feature and three fixes landed, a new uncommitted callback-contract cleanup requiring batch sizes of at least one, and Amit planning to move the broader batching work to a new thread; he also explained existing foreign-partition row-movement and constraint limitations #114 #115.