A potential memory leak on Merge Join when Sort node is not below Materialize node
CREATE TABLE t1 (a text);
CREATE TABLE t2 (a text);
-- make the text a little large by adding 100000000000
INSERT INTO t1 SELECT (100000000000+i%1000)::text FROM
generate_series(0,10000000) i;
-- make the text a little large by adding 100000000000
INSERT INTO t2 SELECT (100000000000+i%10000)::text FROM
generate_series(0,10000000) i;
-- to simplify the explain plan, not strictly necessary
SET max_parallel_workers_per_gather TO 0;
-- these two are necessary so that the problem is triggered
-- these are helpful to use Merge join and avoid materialization
SET enable_hashjoin TO false;
SET enable_material TO false;
-- the join is on a TEXT column
-- when the join is on INT column with a similar setup, I do not observe
this problem
SELECT count(*) FROM t1 JOIN t2 USING (a);
```
The explain output for the query like the following:
```
explain SELECT count(*) FROM t1 JOIN t2 USING (a);
I run your test here with a fix attached.
Can you retake your test with the patch attached?
regards,
Ranier Vilela
Attachments:
fix_possible_memory_leak_merge_join_sort.patchapplication/octet-stream; name=fix_possible_memory_leak_merge_join_sort.patchDownload
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 421afcf47d..12e2fbb2c7 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2657,9 +2657,6 @@ tuplesort_getdatum(Tuplesortstate *state, bool forward,
return false;
}
- /* Ensure we copy into caller's memory context */
- MemoryContextSwitchTo(oldcontext);
-
/* Record abbreviated key for caller */
if (state->sortKeys->abbrev_converter && abbrev)
*abbrev = stup.datum1;
@@ -2676,6 +2673,8 @@ tuplesort_getdatum(Tuplesortstate *state, bool forward,
*isNull = false;
}
+ MemoryContextSwitchTo(oldcontext);
+
return true;
}
Hi,
Thanks for replying so quickly!
I run your test here with a fix attached.
Can you retake your test with the patch attached?
Unfortunately, with the patch, I still see the memory usage increase and
get the OOMs
Thanks,
Onder KALACI
Em qua., 28 de set. de 2022 às 14:24, Önder Kalacı <onderkalaci@gmail.com>
escreveu:
Hi,
Thanks for replying so quickly!
I run your test here with a fix attached.
Can you retake your test with the patch attached?
Unfortunately, with the patch, I still see the memory usage increase and
get the OOMs
Thanks for sharing the result.
regards,
Ranier Vilela