Is a clearer memory lifespan for outerTuple and innerTuple useful?
Hackorum builds and tests every patch posted to the lists, not only commitfest submissions. This is Hackorum's own CI rather than the PostgreSQL project's, and it is still under testing - please report anything that looks wrong.
You can run a PostgreSQL built from this patch straight from Docker, with no checkout and no build:
docker run --rm -p 5432:5432 ghcr.io/hackorum-dev/postgres-patch:t48904psql -h localhost -U postgresBuilt from patchset v3 (message #3), October 06, 2026 at 08:08 AM.
Every patchset is also pushed to a branch of our PostgreSQL fork, so you can check out the same tree CI built. Without a PostgreSQL checkout:
git clone --branch t48904_3 https://github.com/hackorum-dev/postgres.gitIn a checkout you already have, add the fork once:
git remote add hackorum https://github.com/hackorum-dev/postgres.gitthen, for this patchset and every later one:
git fetch hackorum t48904_3 && git checkout t48904_3Patchset v3 (message #3) is on t48904_3
Hi,
When I am working on "shared detoast value"[0]/messages/by-id/87ttoyihgm.fsf@163.com, where I want to avoid
detoast the same datum over and over again, I have to decide which
memory context should be used to hold the detoast value. later I
found I have to use different MemoryContexts for the OuterTuple and
innerTuple since OuterTuple usually have a longer lifespan.
I found the following code in nodeMergeJoin.c which has pretty similar
situation, just that it uses ExprContext rather than MemoryContext.
MergeJoinState *
ExecInitMergeJoin(MergeJoin *node, EState *estate, int eflags)
/*
* we need two additional econtexts in which we can compute the join
* expressions from the left and right input tuples. The node's regular
* econtext won't do because it gets reset too often.
*/
mergestate->mj_OuterEContext = CreateExprContext(estate);
mergestate->mj_InnerEContext = CreateExprContext(estate);
IIUC, we needs a MemoryContext rather than ExprContext in fact. In the
attachment, I just use two MemoryContext instead of the two ExprContexts
which should be less memory and more precise semantics, and works
fine. shall we go in this direction? I attached the 2 MemoryContext in
JoinState rather than MergeJoinState, which is for the "shared detoast
value"[0]/messages/by-id/87ttoyihgm.fsf@163.com more or less.
Attachments:
v1-0001-Use-MemoryContext-instead-of-ExprContext-for-node.patchtext/x-diffDownload+17-16
Andy Fan <zhihuifan1213@163.com> writes:
..., I attached the 2 MemoryContext in
JoinState rather than MergeJoinState, which is for the "shared detoast
value"[0] more or less.
After thinking more, if it is designed for "shared detoast value" patch
(happens on ExecInterpExpr stage), the inner_tuple_memory and
outer_tuple_memory should be attached to ExprContext rather than
JoinState since it is more natual to access ExprConext (compared with
JoinState) in ExecInterpExpr. I didn't attach a new version for this,
any feedback will be appreciated.
--
Best Regards
Andy Fan
Andy Fan <zhihuifan1213@163.com> writes:
Andy Fan <zhihuifan1213@163.com> writes:
..., I attached the 2 MemoryContext in
JoinState rather than MergeJoinState, which is for the "shared detoast
value"[0] more or less.
In order to delimit the scope of this discussion, I attached the 2
MemoryContext to MergeJoinState. Since the code was writen by Tom at
2005, so add Tom to the cc-list.
Hi!
Maybe, the alternative way is using a separate kind of context, say name it
'ToastContext' for all custom data related to Toasted values? What do you
think?
On Sun, Dec 17, 2023 at 4:52 PM Andy Fan <zhihuifan1213@163.com> wrote:
Andy Fan <zhihuifan1213@163.com> writes:
Andy Fan <zhihuifan1213@163.com> writes:
..., I attached the 2 MemoryContext in
JoinState rather than MergeJoinState, which is for the "shared detoast
value"[0] more or less.In order to delimit the scope of this discussion, I attached the 2
MemoryContext to MergeJoinState. Since the code was writen by Tom at
2005, so add Tom to the cc-list.--
Best Regards
Andy Fan
--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company
https://postgrespro.ru/
Nikita Malakhov <hukutoc@gmail.com> writes:
Hi!
Maybe, the alternative way is using a separate kind of context, say name it
'ToastContext' for all custom data related to Toasted values? What do
you think?
That should be a candidate. The latest research makes me think the
'detoast_values' should have the same life cycles as tts_values, so the
memory should be managed by TupleTuleSlot (rather than ExprContext) and
be handled in ExecCopySlot / ExecClearSlot stuff.
In TupleTableSlot we already have a tts_mctx MemoryContext, reusing it
needs using 'pfree' to free the detoast values and but a dedicated
memory context pays more costs on the setup, but a more efficient
MemoryContextReset.
On Sun, Dec 17, 2023 at 4:52 PM Andy Fan <zhihuifan1213@163.com> wrote:
Andy Fan <zhihuifan1213@163.com> writes:
Andy Fan <zhihuifan1213@163.com> writes:
..., I attached the 2 MemoryContext in
JoinState rather than MergeJoinState, which is for the "shared detoast
value"[0] more or less.In order to delimit the scope of this discussion, I attached the 2
MemoryContext to MergeJoinState. Since the code was writen by Tom at
2005, so add Tom to the cc-list.
However this patch can be discussed seperately.
--
Best Regards
Andy Fan