fork/exec problem: DynaHashCxt

Started by Claudio Natoliabout 22 years ago5 messages
#1Claudio Natoli
claudio.natoli@memetrics.com

Hi all,

I'm working on getting BackendFork converted to a fork/exec model, and have
hit the following wall.

All the ShmemInitHash structures are allocated using DynaHashCxt. Clearly,
this context is going to have to be shared across backends in the fork/exec
case, but I can't see a non-trivial way of doing so.

Any ideas? (or have I mis-diagnosed the problem completely)

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>
#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Claudio Natoli (#1)
Re: [HACKERS] fork/exec problem: DynaHashCxt

Claudio Natoli <claudio.natoli@memetrics.com> writes:

All the ShmemInitHash structures are allocated using DynaHashCxt.

I'm not sure if you're confusing backend-local hashes with shared
hashes, or hash control headers with the actual shared data. But
the above is a false statement. DynaHashCxt is not shared.

Shared hashes are a bit tricky because there is a backend-local
structure that has a pointer into the shared memory --- is that
what's confusing you?

Keep in mind that this code did work in a fork/exec context not
so many moons ago. If you think you see a showstopper, it's a
relatively recent change and can be undone.

regards, tom lane

#3Claudio Natoli
claudio.natoli@memetrics.com
In reply to: Tom Lane (#2)
Re: [HACKERS] fork/exec problem: DynaHashCxt

I'm not sure if you're confusing backend-local hashes with shared
hashes, or hash control headers with the actual shared data. But
the above is a false statement. DynaHashCxt is not shared.

No, wasn't confused over that. Was confused over something else though :-)

Shared hashes are a bit tricky because there is a backend-local
structure that has a pointer into the shared memory --- is that
what's confusing you?

That's pretty much right on the mark, and the heart of the problem I
suspect.

So this means we'll have to pull relHash out of the shared FreeSpaceMap
structure and make it a variable in it's own right? [Same goes for lockHash
and proclockHash in the LockMethod structure reference by "LockTable (lock
method table)"]

Keep in mind that this code did work in a fork/exec context not
so many moons ago. If you think you see a showstopper, it's a
relatively recent change and can be undone.

Keeping this firmly in mind, trust me.

Thanks,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>
#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Claudio Natoli (#3)
Re: [HACKERS] fork/exec problem: DynaHashCxt

Claudio Natoli <claudio.natoli@memetrics.com> writes:

So this means we'll have to pull relHash out of the shared FreeSpaceMap
structure and make it a variable in it's own right?

Hm. The freespace.c code is relatively new and might not be jumping
through all of the hoops it should be jumping through. My recollection
of the older code is that the logic was like "create or attach to shared
memory structure named 'foo', if not create then initialize the shared
structure". I'll take the blame if freespace.c doesn't do this right...

[Same goes for lockHash and proclockHash in the LockMethod structure
reference by "LockTable (lock method table)"]

The lock code *should* be doing this right, unless I've totally
forgotten the modification history ...

regards, tom lane

#5Claudio Natoli
claudio.natoli@memetrics.com
In reply to: Tom Lane (#4)
Re: [HACKERS] fork/exec problem: DynaHashCxt

Claudio Natoli <claudio.natoli@memetrics.com> writes:

So this means we'll have to pull relHash out of the shared FreeSpaceMap
structure and make it a variable in it's own right?

Hm. The freespace.c code is relatively new and might not be jumping
through all of the hoops it should be jumping through. My
recollectionof the older code is that the logic was like "create or
attach to shared memory structure named 'foo', if not create then

initialize the shared

structure". I'll take the blame if freespace.c doesn't do this right...

The way I see it:
* Postmaster call InitFreeSpaceMap
* When a fork/exec backend starts, it'll be able to access the
FreeSpaceMap struct via shared memory
* But this very same struct contains a pointer (relHash) to a struct
which, in the fork/exec case, is valid only in the postmaster (ie. the hashp
HTAB struct created in the postmaster's DynaHashCxt context and returned by
hash_create/ShmemInitHash)
[of course, this HTABs struct htcl value will point to shared memory, but
it is a moot point by this stage]

[Same goes for lockHash and proclockHash in the LockMethod structure
reference by "LockTable (lock method table)"]

The lock code *should* be doing this right, unless I've totally
forgotten the modification history ...

AFAICS, the reasoning above applies to these hashes. Of course, I'm more
than ready to be corrected on this reasoning...

Cheers,
Claudio

--- 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
<a
href="http://www.memetrics.com/emailpolicy.html">http://www.memetrics.com/em
ailpolicy.html</a>