Re: Threads vs Processes (was: NuSphere and PostgreSQL for window

Claudio Natoli <claudio.natoli@memetrics.com> writes:

How are you dealing with the issue of wanting some static variables to
be per-thread and others not?

To be perfectly honest, I'm still trying to familiarize myself with the code
sufficiently well so that I can tell which variables need to be per-thread
and which are shared (and, in turn, which of these need to be protected from
concurrent access).

Well, the first-order approximation would be to duplicate the current
fork semantics: *all* static variables are per-thread, and should be
copied from the parent thread at thread creation. If there is some
reasonably non-invasive way to do that, we'd have a long leg up on the
problem.

regards, tom lane

Tom Lane wrote:

Claudio Natoli <claudio.natoli@memetrics.com> writes:

How are you dealing with the issue of wanting some static variables to
be per-thread and others not?

To be perfectly honest, I'm still trying to familiarize myself with the code
sufficiently well so that I can tell which variables need to be per-thread
and which are shared (and, in turn, which of these need to be protected from
concurrent access).

Well, the first-order approximation would be to duplicate the current
fork semantics: *all* static variables are per-thread, and should be
copied from the parent thread at thread creation. If there is some
reasonably non-invasive way to do that, we'd have a long leg up on the
problem.

Hmm.. I was looking for some fast tutorials on thread local storage. I found
this one..
http://publib16.boulder.ibm.com/pseries/en_US/aixprggd/genprogc/thread_specific_data.htm

Basically, in a process we are free to declare as many globals as we can.
Converting them to thread local is not an easy job because each variable would
need it's own key and there is limit on how many keys can be allocated.

One thing that can be done is to arrange all globals/statics in a structure and
make that structure thread local. We need to change all invocations of any of
those variables to use a pointer. We just need only one global variable. And
some macro trickery possibly so that we can extend that structure easily and
automatically.

Upshot is duplicating environment is easy. We need to do a huge memcpy and any
specific depp copy of strings on thread creation. Besides even in process model,
this kind of initialization will allow to put all variables on heap instead of
stack. But then we need to add initialization code explicitly.

Something like int a=10; can not be added just like that.

If globals are less than 100 in numbers, I think it should be reasonably blind
job of converting them to a structure type stuff. Don't know really though. My
estimations are always 10% of what it takes..:-)

I hope I got it correct..

Shridhar

Tom Lane wrote:

Claudio Natoli <claudio.natoli@memetrics.com> writes:

How are you dealing with the issue of wanting some static variables to
be per-thread and others not?

To be perfectly honest, I'm still trying to familiarize myself with the code
sufficiently well so that I can tell which variables need to be per-thread
and which are shared (and, in turn, which of these need to be protected from
concurrent access).

No. Not protected from concurrent access. Each thread must have it's own
copy.

Well, the first-order approximation would be to duplicate the current
fork semantics: *all* static variables are per-thread, and should be
copied from the parent thread at thread creation. If there is some
reasonably non-invasive way to do that, we'd have a long leg up on the
problem.

There is a declspec(thread) that makes a global variable per-thread.
AFAIK it uses linker magic to replace the actual memory accesses with
calls to TlsAlloc() etc. Note that declspec(thread) doesn't work from
within dynamic link libraries, but that shouldn't be a big problem.

--
Manfred

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

One thing that can be done is to arrange all globals/statics in a
structure and make that structure thread local.

That's about as far from "non-invasive" as I can imagine :-(

I really, really want to avoid doing anything like the above, because it
would force us to expose to the whole backend many data structures and
state variables that are currently local to individual .c files. That
complicates understanding and debugging tremendously, not to mention
slowing the edit/compile/debug cycle when you are changing such
structures.

regards, tom lane

Both Microsoft and windows compilers support thread local storage. *If*
you guys go with the threading model and *if* it does not introduce any
serious portability issues with gcc (big ifs, and I'm not familiar with
gcc), than IMO TLS is really the way to go. I don't think any
reorganization of postgres's static variables is necessary. TLS is
implemented in the win32 API, not the C Libs, so by giving up the syntax
sugar you can make direct calls and keep compiler independence in win32.

Microsoft syntax is __desclspec(thread) and Borland syntax is simply
__thread. All TLS variables *must* be static (or implicitly static
through extern, i.e. no 'auto' variables) and their addresses can not be
assumed to be constant.

Taking addresses of TLS variables should be considered illegal, as well
as pointers to TLS variables. Another gotcha is that DLLs that have
__thread variables will GPF if loaded with LoadLibrary (they should be
static linked). Of course, pg does not use dlls, but it's worth noting.

Merlin

Actually you can use a DLL with LoadLibrary as long as you do the following.

When a process uses load-time linking with this DLL, the entry-point
function is sufficient to manage the thread local storage. Problems can
occur with a process that uses run-time linking because the entry-point
function is not called for threads that exist before the LoadLibrary
function is called, so TLS memory is not allocated for these threads. The
following example solves this problem by checking the value returned by the
TlsGetValue function and allocating memory if the value indicates that the
TLS slot for this thread is not set.

LPVOID lpvData;

// Retrieve a data pointer for the current thread.

lpvData = TlsGetValue(dwTlsIndex);

// If NULL, allocate memory for this thread.

if (lpvData == NULL)
{
lpvData = (LPVOID) LocalAlloc(LPTR, 256);
if (lpvData != NULL)
TlsSetValue(dwTlsIndex, lpvData);
}

Unless gcc has extension as Borland and Microsoft do you will have to
utilize the API and not the compiler/linker customizations that make access
variables declared as __declspec(thread) or __thread under Borland.

Keith

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Merlin Moncure
Sent: Thursday, September 25, 2003 2:49 PM
To: Tom Lane
Cc: pgsql-hackers@postgresql.org; pgsql-hackers-win32@postgresql.org; Bruce
Momjian; Shridhar Daithankar; Claudio Natoli
Subject: Re: [HACKERS] Threads vs Processes

Both Microsoft and windows compilers support thread local storage. *If* you
guys go with the threading model and *if* it does not introduce any serious
portability issues with gcc (big ifs, and I'm not familiar with gcc), than
IMO TLS is really the way to go. I don't think any reorganization of
postgres's static variables is necessary. TLS is implemented in the win32
API, not the C Libs, so by giving up the syntax sugar you can make direct
calls and keep compiler independence in win32.

Microsoft syntax is __desclspec(thread) and Borland syntax is simply
__thread. All TLS variables *must* be static (or implicitly static through
extern, i.e. no 'auto' variables) and their addresses can not be assumed to
be constant.

Taking addresses of TLS variables should be considered illegal, as well as
pointers to TLS variables. Another gotcha is that DLLs that have __thread
variables will GPF if loaded with LoadLibrary (they should be static
linked). Of course, pg does not use dlls, but it's worth noting.

Merlin

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:

All TLS variables *must* be static (or implicitly static
through extern, i.e. no 'auto' variables)

I assume you mean static as in not-auto, rather than static as in
not-global. Otherwise we have a problem here.

and their addresses can not be
assumed to be constant.

Surely the addresses can be assumed constant within a thread. Otherwise
we have a problem here too.

Taking addresses of TLS variables should be considered illegal,

Sorry, no can accept that restriction.

regards, tom lane

On Thursday, September 25, 2003, at 10:03 AM, Tom Lane wrote:

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

One thing that can be done is to arrange all globals/statics in a
structure and make that structure thread local.

That's about as far from "non-invasive" as I can imagine :-(

I really, really want to avoid doing anything like the above, because
it
would force us to expose to the whole backend many data structures and
state variables that are currently local to individual .c files. That
complicates understanding and debugging tremendously, not to mention
slowing the edit/compile/debug cycle when you are changing such
structures.

Another option would be to create thread local hashtable or other lookup
structure to which you would register a structure for a particular .c
file
or group of files.

You could then define the structures you need locally without affecting
other parts of the codebase.

Myron Scott

Tom Lane wrote:

I assume you mean static as in not-auto, rather than static as in
not-global. Otherwise we have a problem here.

[...]

Surely the addresses can be assumed constant within a thread. Otherwise
we have a problem here too.

[...]

Taking addresses of TLS variables should be considered illegal,

Sorry, no can accept that restriction.

I think you are okay on all 3 fronts,
from http://gcc.gnu.org/onlinedocs/gcc/Thread-Local.html#Thread-Local :

"The __thread specifier may be used alone, with the extern or static
specifiers, but with no other storage class specifier. When used with
extern or static, __thread must appear immediately after the other
storage class specifier."

and

"When the address-of operator is applied to a thread-local variable, it
is evaluated at run-time and returns the address of the current thread's
instance of that variable. An address so obtained may be used by any
thread. When a thread terminates, any pointers to thread-local variables
in that thread become invalid."

Also see "ISO/IEC 9899:1999 Edits for Thread-Local Storage" :
http://gcc.gnu.org/onlinedocs/gcc/C99-Thread-Local-Edits.html#C99%20Thread-Local%20Edits

and ELF Handling For Thread-Local Storage,
http://people.redhat.com/drepper/tls.pdf may be of interest.

Cheers,
Kurt.

Another option would be to create thread local hashtable or other lookup
structure to which you would register a structure for a particular .c
file or group of files.

You could then define the structures you need locally without
affecting other parts of the codebase.

Myron Scott

A slight variant on this idea would be to mask the variables themselves,
using macrology on a hash keyed on the threadID.

To illustrate:
MyProc->errType

would, on a threaded system, become something like:
((PGProc*)ThreadLocalHash(MyProc,GetCurrentThreadID()))->errType

I'm imagining that an approach like this would fit in very nicely with the
existing code. Not the most performance friendly solution however (and I'm
guessing that there might be a place or two where this might be important
:-).

Just a thought,
Claudio

--- 
WE HAVE MOVED - PLEASE NOTE OUR NEW CONTACT DETAILS: 
THE BASEMENT, 33 EWELL STREET, BALMAIN NSW 2041 
TEL: +61 2 9555 1544 FAX: +61 2 9555 6911 
Certain disclaimers and policies apply to all email sent from Memetrics.
For the full text of these disclaimers and policies see 
http://www.memetrics.com/emailpolicy.html

"When the address-of operator is applied to a thread-local variable, it
is evaluated at run-time and returns the address of the current thread's
instance of that variable. An address so obtained may be used by any
thread. When a thread terminates, any pointers to thread-local variables
in that thread become invalid."

Bummer, I would have thought one advantage of using TLS must surely be memory
protection ? So the only for pg useful usage for TLS seems to be "__declspec(thread)"
and "__declspec(thread) static" (both for stuff that do not need runtime
preinitialization).

Maybe the techniques of electric fence could be used for protecting the shmem
at least a little bit.

Andreas

Tom Lane wrote:
"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:

All TLS variables *must* be static (or implicitly static
through extern, i.e. no 'auto' variables)

I assume you mean static as in not-auto, rather than static as in
not-global. Otherwise we have a problem here.

Yes, you are correct.

and their addresses can not be
assumed to be constant.

Surely the addresses can be assumed constant within a thread.

Otherwise

we have a problem here too.

Quoting from the MSDN:
The address of a thread local object is not considered constant, and any
expression involving such an address is not considered a constant
expression. In standard C, the effect of this is to forbid the use of
the address of a thread local variable as an initializer for an object
or pointer. For example, the following code will be flagged as an error
by the C compiler:
#define Thread __declspec( thread )
Thread int tls_i;
int *p = &tls_i; //This will generate an error in C.
<end>

(Note this does not apply to C++)

Taking addresses of TLS variables should be considered illegal,

Sorry, no can accept that restriction.

I thought not. I believe if TLS variables are detail managed through
the win32 API, some of these problems can be avoided (after all, these
examples are for the Microsoft compiler).

Merlin

Tom Lane wrote:

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

One thing that can be done is to arrange all globals/statics in a
structure and make that structure thread local.

That's about as far from "non-invasive" as I can imagine :-(

I know.

I have following half baked solution. Obviously it won't be as good and
automatic as __declspec(thread). But still... Just for starters

1. Put all static/global variables in .h file. No more in .c files
2. Encapsulate them in a #define.
3. Create a new header files which includes lot of macro invocations. That way
we expand a macro definition, the structure definition changes on recompile.

Like in a.h
a.h
---
#define module1 int a;\
int b;
---

b.h
---
typedef struct
{
module1
module2
} globalDataStructure

---

Whoever includes a.h earlier, needs to include b.h now. That way things will
flow smoothly. This could be optional but I would rather not include a.h anymore
as a good practice.

4. Add a macro definition for initialization function. The global initialization
function will again invoke macro implementations which are free to redirect it
to any function of choice.

That way we can maintain variables in modules that they are but still put them
in a single compile unit when they get compiled. Only one file b.h, need to be
maintained for addition of new files. The variables would be taken care of in
the respective modules. ( This could nicely be done in C++ using classes but
thats a
altogether different subject.)

The changes to actual C code obviously need to follow.

I agree that's messy and non transparent. but it still retains distributed
nature of variable declarations and maintenance thereof..

But I feel we are missing the point. Last time threads for unix builds were
discussed, one of the good suggestion was to use them to distribute load across
CPU's rather than using thread per connection. Personally I don't think
postgresql is missing lot of performance by using process/connection rather than
thread/connection(Except for slowaris and windows perhaps).

Even on windows, the time taken for a new connection is unlikely to be an issue,
I feel.

We really don't need threads to replace existing functionality. That would be
dog work.

If we are writing new functionality, we can do it clean and portable way rather
than fiddling with existing code. Changing working code is always a messy
business irrespective of technicalities involved. As a support engineer, I can
attest that.

Shridhar

"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:

Tom Lane wrote:

Surely the addresses can be assumed constant within a thread.
Otherwise we have a problem here too.

Quoting from the MSDN:
The address of a thread local object is not considered constant, and any
expression involving such an address is not considered a constant
expression.

Ah. That's probably reasonable. Still a bit of a PITA for us, as there
are various places that do give a static variable an initializer
pointing to another static. But that could be worked around I think.
I thought you were saying that the compiler would forbid taking a TLS
variable's address even at runtime.

regards, tom lane

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

We really don't need threads to replace existing functionality. That
would be dog work.

No, that's not the point at all. The problem we are facing at the
moment with the Windows port is lack of fork(), which means there's
no way for separate-subprocess backends to inherit variable values
from the postmaster. Bruce has been trying to fix that by having the
subprocesses somehow reload or re-deduce all those variables; which
is messy, bug-prone, and probably race-condition-prone too. In a
threaded implementation it would maybe be relatively easy to initialize
a new thread's TLS by copying the postmaster thread's TLS, in which case
a whole pile of as-yet-unwritten Windows-only code won't be needed.

regards, tom lane

On Friday 26 September 2003 20:19, Tom Lane wrote:

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

We really don't need threads to replace existing functionality. That
would be dog work.

No, that's not the point at all. The problem we are facing at the
moment with the Windows port is lack of fork(), which means there's
no way for separate-subprocess backends to inherit variable values
from the postmaster. Bruce has been trying to fix that by having the
subprocesses somehow reload or re-deduce all those variables; which
is messy, bug-prone, and probably race-condition-prone too. In a
threaded implementation it would maybe be relatively easy to initialize
a new thread's TLS by copying the postmaster thread's TLS, in which case
a whole pile of as-yet-unwritten Windows-only code won't be needed.

Umm.. I understand child process created by createProcess does not inherit
variable values from parent process. That's where problem originates..

We can simply create a registry key that would contain shared memory id from
where a child process should get the variable values.

And that would need initialization function I talked about earlier. And since
anyways TLS->TLS copy is still needed anyways, I think this approach can
still save us dealing with threads.

God.. it doesn't get any less messy..I hope this is of some value..

Shridhar

Tom Lane wrote:

"Merlin Moncure" <merlin.moncure@rcsonline.com> writes:

Tom Lane wrote:

Surely the addresses can be assumed constant within a thread.
Otherwise we have a problem here too.

Quoting from the MSDN:
The address of a thread local object is not considered constant, and any
expression involving such an address is not considered a constant
expression.

Ah. That's probably reasonable. Still a bit of a PITA for us, as there
are various places that do give a static variable an initializer
pointing to another static. But that could be worked around I think.
I thought you were saying that the compiler would forbid taking a TLS
variable's address even at runtime.

Tom,

you wrote you wouldn't like the idea of a struct representing the now
global variables for a thread, because this would mean that every module
would need to access it, and any change of a module-local variable would
affect the complete backend.

This could be worked around:
If that global struct is just a list of pointers to memory blocks, each
block representing the opaque local data of a module, this can be
avoided. This could be duplicated easily for thread creation, if zwo
ints representing mem block size and size to copy for each block is
included. Additional pointers for creation and cleanup functions could
help for more special initializations.

This would make the thing independent of fancy compiler features and
platforms.

Regards,
Andreas

Tom Lane wrote:

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> writes:

We really don't need threads to replace existing functionality. That
would be dog work.

No, that's not the point at all. The problem we are facing at the
moment with the Windows port is lack of fork(), which means there's
no way for separate-subprocess backends to inherit variable values
from the postmaster. Bruce has been trying to fix that by having the
subprocesses somehow reload or re-deduce all those variables; which
is messy, bug-prone, and probably race-condition-prone too. In a
threaded implementation it would maybe be relatively easy to initialize
a new thread's TLS by copying the postmaster thread's TLS, in which case
a whole pile of as-yet-unwritten Windows-only code won't be needed.

I haven't said much in this thread yet because I wasn't sure what to
say.

I worked 2-3 weeks on Win32 back in March/April, and a few days last
month getting it to compile again, and for the interfaces/clients to run
under Win32.

I haven't had any time to work on the fork/exec, though I now have a few
weeks to spend on it, so it isn't that I have been trying to get
fork/exec working, and failing, it is that I haven't even tried lately.

My plan is to pass a few values to the child via the command line:

#ifdef EXEC_BACKEND
Assert(UsedShmemSegID != 0 && UsedShmemSegAddr != NULL);
/* database name at the end because it might contain commas */
snprintf(pbuf, NAMEDATALEN + 256, "%d,%d,%d,%p,%s", port->sock, canAcceptConnections(),
UsedShmemSegID, UsedShmemSegAddr, port->database_name);
av[ac++] = pbuf;
#else

and pass the GUC settings via a special binary file. (Those are already
in main CVS.) The other values I plan to regenerate in the child the
same way the postmaster does it at initialization time. The easy part
of that is that I only have to worry about postmaster variables. All my
current fork/exec work is marked by #ifdef EXEC_BACKEND in current CVS,
so it can be easily ripped out.

One solution is for me to continue with this in the Win32 CVS version
until I have fork/exec() working on Unix, then test on Win32. I think
that could be done in a few weeks, if not less.

Another solution, already mentioned, is to use threads and TLS. This is
what SRA's code uses. I know SRA wants to contribute that code back to
the community, so I can ask them to see if they are ready to release it.
That would show us all the changes needed to do threading. Their code
is based on 7.3.X, rather than PeerDirect's which is based on 7.2.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

One solution is for me to continue with this in the Win32 CVS version
until I have fork/exec() working on Unix, then test on Win32. I think
that could be done in a few weeks, if not less.

Another solution, already mentioned, is to use threads and TLS. This is
what SRA's code uses. I know SRA wants to contribute that code back to
the community, so I can ask them to see if they are ready to release it.

If you are willing to expend the effort, I think it would be worth the
time to pursue both approaches. We don't yet have enough info to decide
which one will be cleaner, so we need to push forward on both until we
can make a realistic comparison.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

One solution is for me to continue with this in the Win32 CVS version
until I have fork/exec() working on Unix, then test on Win32. I think
that could be done in a few weeks, if not less.

Another solution, already mentioned, is to use threads and TLS. This is
what SRA's code uses. I know SRA wants to contribute that code back to
the community, so I can ask them to see if they are ready to release it.

If you are willing to expend the effort, I think it would be worth the
time to pursue both approaches. We don't yet have enough info to decide
which one will be cleaner, so we need to push forward on both until we
can make a realistic comparison.

I think I know enough to get the fork/exec working, particularly because
most of the work can be tested under Unix. I don't know enough to get
the threads working, and I have had no offers of help since I started.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Friday, September 26, 2003 9:27 AM
To: Bruce Momjian
Cc: Shridhar Daithankar; pgsql-hackers@postgresql.org;
pgsql-hackers-win32@postgresql.org
Subject: Re: [HACKERS] Threads vs Processes

Bruce Momjian <pgman@candle.pha.pa.us> writes:

One solution is for me to continue with this in the Win32

CVS version

until I have fork/exec() working on Unix, then test on

Win32. I think

that could be done in a few weeks, if not less.

Another solution, already mentioned, is to use threads and

TLS. This

is what SRA's code uses. I know SRA wants to contribute that code
back to the community, so I can ask them to see if they are

ready to

release it.

If you are willing to expend the effort, I think it would be
worth the time to pursue both approaches. We don't yet have
enough info to decide which one will be cleaner, so we need
to push forward on both until we can make a realistic comparison.

I think the ideal situation would be a server that both threads and
forks.

Sometimes, we want the server to take on a new personality (with the
rights of the attaching user). In such a case, threading is not a
sufficient answer. Also, a forked process is a bit safer (though you
can put a try/catch around threads).

For performance with multiple queries from a single user, threads are
going to be faster than forking. I think that the best model will be a
server that does both.

I know that there is some aversion to using C++, but the ACE framework
offers a consistent threading model that works for just about every
computer under the sun.
http://www.cs.wustl.edu/~schmidt/ACE.html

The license is basically BSD (it is called ACE, but it works exactly
like a BSD license). With ACE, you program to a single API, and the
code works the same on every platform with a simple recompile. It might
even be worthwhile to use the ACE higher level components to create a
server that supports multiple models of connection and threading.

We really don't need threads to replace existing functionality. That
would be dog work.

No, that's not the point at all. The problem we are facing at the
moment with the Windows port is lack of fork(), which means there's
no way for separate-subprocess backends to inherit variable values
from the postmaster. Bruce has been trying to fix that by having the
subprocesses somehow reload or re-deduce all those variables; which
is messy, bug-prone, and probably race-condition-prone too. In a
threaded implementation it would maybe be relatively easy to initialize
a new thread's TLS by copying the postmaster thread's TLS, in which case
a whole pile of as-yet-unwritten Windows-only code won't be needed.

Kepp in mind though all the cool things that could be done if we had
thread capabilities. eg. evaluating different subexpressings on
fdifferent cpuis for the one query, etc.

Chris