Advice wanted on backend memory management
I want to change hashjoin's use of a fixed-size overflow area for tuples
that don't fit into the hashbucket they ought to go in. Since it's
always possible for an improbably large number of tuples to hash into the
same hashbucket, the overflow area itself can overflow; without the
ability to recover from that, hashjoin is inherently unreliable.
So I think this is an important thing to fix.
To do this, I need to be able to allocate chunks of space that I will
later want to give back all at once (at the end of a hash pass).
Seems to me like a job for palloc and a special memory context ---
but I see no way in mcxt.h to create a new memory context. How do
I do that? Also, I'd want the new context to be a "sub-context" of
the regular execution context, in the sense that it should automatically
get released if we exit via elog(ERROR). What are the appropriate
calls to be using for this? If there's documentation about this stuff,
I haven't found it :-(
regards, tom lane
Tom Lane wrote:
I want to change hashjoin's use of a fixed-size overflow area for tuples
that don't fit into the hashbucket they ought to go in. Since it's
always possible for an improbably large number of tuples to hash into the
same hashbucket, the overflow area itself can overflow; without the
ability to recover from that, hashjoin is inherently unreliable.
So I think this is an important thing to fix.To do this, I need to be able to allocate chunks of space that I will
later want to give back all at once (at the end of a hash pass).
Seems to me like a job for palloc and a special memory context ---
but I see no way in mcxt.h to create a new memory context. How do
I do that? Also, I'd want the new context to be a "sub-context" of
No way :(
StartPortalAllocMode could help but - portalmem.c:
/*
* StartPortalAllocMode
* Starts a new block of portal heap allocation using mode and limit;
* the current block is disabled until EndPortalAllocMode is called.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I'm unhappy with this allocation block stacking for quite long time :(
Try to pfree chunks "by hand".
Vadim
Vadim Mikheev <vadim@krs.ru> writes:
Try to pfree chunks "by hand".
Yeah, that's what I'm trying to avoid. That would basically mean
duplicating the logic that's in aset.c, which is pretty silly...
After some more looking around, it looks like I could create a
"portal" as is done in vacuum or spi. But portals seem to have
a heckuva lot of features that I don't understand the uses for.
Anyone have any comments or documentation about them?
regards, tom lane
Import Notes
Reply to msg id not found: YourmessageofWed05May1999091813+0800372F9C55.C0BADB02@krs.ru | Resolved by subject fallback
Your e-mail did not arrive at its intended destination. You need to
send it to Michael J. Davis, not Michael Davis.
From: Vadim Mikheev <vadim @ krs.ru> on 05/04/99 09:18 PM
To: Tom Lane <tgl @ sss.pgh.pa.us>@SMTP@EXCHANGE
cc: pgsql-hackers @ postgreSQL.org@SMTP@EXCHANGE
Subject: Re: [HACKERS] Advice wanted on backend memory
management
Tom Lane wrote:
I want to change hashjoin's use of a fixed-size overflow area for
tuples
that don't fit into the hashbucket they ought to go in. Since
it's
always possible for an improbably large number of tuples to hash
into the
same hashbucket, the overflow area itself can overflow; without
the
ability to recover from that, hashjoin is inherently unreliable.
So I think this is an important thing to fix.To do this, I need to be able to allocate chunks of space that I
will
later want to give back all at once (at the end of a hash pass).
Seems to me like a job for palloc and a special memory context ---
but I see no way in mcxt.h to create a new memory context. How do
I do that? Also, I'd want the new context to be a "sub-context"
of
No way :(
StartPortalAllocMode could help but - portalmem.c:
/*
* StartPortalAllocMode
* Starts a new block of portal heap allocation using mode and
limit;
* the current block is disabled until EndPortalAllocMode is
called.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I'm unhappy with this allocation block stacking for quite long time
:(
Try to pfree chunks "by hand".
Vadim
Import Notes
Resolved by subject fallback