No title
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg). Let me state the following goals:
a. Thread-safeness, where it makes a difference, is generally thought to
be a performance hit, so the user needs to have a choice to use
thread-safe libraries or not.
b. The user needs to be able to make the choice at the time he builds his
application, *not* at the time the PostgreSQL distribution is built or
installed.
Clearly, a thread-safe ecpg library is always going to be significantly
different from the "normal" one, with all the mutex things that get pulled
in, so it seems reasonable to always offer a libecpg_r alongside the
libecpg.
The question is whether a libpq_r should be provided if libpq is
thread-safe by default (no *_r functions, libc_r, or special flags). I
think yes. It could be a symlink, so it doesn't really waste space. But
it would convenience users: Those who want to be sure to always link
against a thread-safe version can point to libpq_r and don't have to
create complicated detection mechanisms. Those who know that their system
is thread-safe by default can simply use libpq to follow that convention.
And of course it creates consistency with libecpg_r and does not bother
the user with complicated internal artifacts.
A final note on the name of the configure option, --with-threads. First,
it does not control an external package but an internal feature, so it
should be --enable-. Secondly, it does not use threads, only enable
thread-safeness. So --enable-thread-safe might be a better name. Or if
you want to be more precise, --enable-thread-safe-client. The latter is
what MySQL uses, in case anyone cares about that.
--
Peter Eisentraut peter_e@gmx.net
Peter Eisentraut <peter_e@gmx.net> writes:
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg). Let me state the following goals:
Sounds good to me. Do you have time to make these things happen?
regards, tom lane
--On Tuesday, July 22, 2003 11:11:33 -0400 Tom Lane <tgl@sss.pgh.pa.us>
wrote:
Peter Eisentraut <peter_e@gmx.net> writes:
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg). Let me state the following goals:Sounds good to me. Do you have time to make these things happen?
Please see my post to -hackers last week about -K[p]thread on UnixWare.
There
are some issues there for us.
Also, Peter, can you look at the forwards I've made from Kean Johnson of
SCO on
-Patches about the OSR5 patch, and DT_SONAME?
Thanks,
LER
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On 22 Jul 2003 at 16:55, Peter Eisentraut wrote:
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg). Let me state the following goals:a. Thread-safeness, where it makes a difference, is generally thought to
be a performance hit, so the user needs to have a choice to use
thread-safe libraries or not.
On linux and freeBSD, that is not an issue. I can attest that from my
experiments with a webserver. Thread function calls often time less than or of
the order of 1us even with CPU capped.
I would really like to know for what platforms, locking mutex while selecting
from connection list or some such object, is going to be such a great
performance hit.
Performance degradation in threaded programs comes from contentions. It goes
for frequency of contention and number of threads fighting over it.
I doubt any threaded ecpg program would reach that level of contention anytime.
If a lock canbe obtained/released in less than 10us and subsequent database
query is going to take at least a ms, IMO that performance degradation is not
worth that much trouble.
But that is for linux and freeBSD. What other platforms have serious thread
issues?
Clearly, a thread-safe ecpg library is always going to be significantly
different from the "normal" one, with all the mutex things that get pulled
in, so it seems reasonable to always offer a libecpg_r alongside the
libecpg.
I would say, it should be thread-safe by default. No point polluting possible
linkages.
I repeat what I have said earlier. If there are two libraries A using libecpg_r
and B, using libecpg, then program linking against both of them is going to
have tough time living with symbol conflicts.
I suppose problem will be reproducible even under freeBSD if you try to create
a postgresql function in C which uses threads. Link the library against libc_r
and link postgresql against libc. It would run into problems.
I am just stating my experiences.I might have missed solution to this problem.
But overall I like GNU libc approach of everything thread safe by default. If
thread performance is an issue, then it should be improved. Not worked around
with two libraries.
Just a thought..
Bye
Shridhar
--
2180, U.S. History question: What 20th Century U.S. President was almost impeached and what office did he later hold?
Peter Eisentraut wrote:
I've been thinking some about how exactly to provide the new option of
thread-safe clients (libpq, ecpg). Let me state the following goals:a. Thread-safeness, where it makes a difference, is generally thought to
be a performance hit, so the user needs to have a choice to use
thread-safe libraries or not.
OK.
b. The user needs to be able to make the choice at the time he builds his
application, *not* at the time the PostgreSQL distribution is built or
installed.
Agreed.
Clearly, a thread-safe ecpg library is always going to be significantly
different from the "normal" one, with all the mutex things that get pulled
in, so it seems reasonable to always offer a libecpg_r alongside the
libecpg.
True. And we have lots of platforms that we don't know how to enable
thread-safeness.
The question is whether a libpq_r should be provided if libpq is
thread-safe by default (no *_r functions, libc_r, or special flags). I
think yes. It could be a symlink, so it doesn't really waste space. But
it would convenience users: Those who want to be sure to always link
against a thread-safe version can point to libpq_r and don't have to
create complicated detection mechanisms. Those who know that their system
is thread-safe by default can simply use libpq to follow that convention.
And of course it creates consistency with libecpg_r and does not bother
the user with complicated internal artifacts.
I think adding a libpq_r on a platform that doesn't use *_r libraries is
just too confusing.
A final note on the name of the configure option, --with-threads. First,
it does not control an external package but an internal feature, so it
should be --enable-. Secondly, it does not use threads, only enable
thread-safeness. So --enable-thread-safe might be a better name. Or if
you want to be more precise, --enable-thread-safe-client. The latter is
what MySQL uses, in case anyone cares about that.
Peter, you are good with these subtle distinctions. How about
--enable-thread-safeness?
I have an idea. Seems we have two issues, libpq and ecpg.
For ecpg, we need a command-line flag to specify if we want threading in
the application.
For libpq and libpecpg, we will just produce whatever therading library
should be created on that platform, meaning either just one library, or
one with _r and one without.
So we tell them, if you want the ability to do threaded libpq and ecpg,
use the --enable-thread-safeness configure flag. If it succeeds, it
means we know how to do it for that operating system, and you get
threading capability. If it fails, you don't have threading.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073Larry Rosenman writes:
Please see my post to -hackers last week about -K[p]thread on UnixWare.
There are some issues there for us.
The take-home message from those posts is that if you use non-thread-safe
code in threaded applications, bad things will happen. We knew that. ;-)
As far as mixing code compiled one way with code compiled the other way,
there is no problem, at least not more than on any other platform.
--
Peter Eisentraut peter_e@gmx.net
--On Wednesday, July 23, 2003 12:12:45 +0200 Peter Eisentraut
<peter_e@gmx.net> wrote:
Larry Rosenman writes:
Please see my post to -hackers last week about -K[p]thread on UnixWare.
There are some issues there for us.The take-home message from those posts is that if you use non-thread-safe
code in threaded applications, bad things will happen. We knew that. ;-)
As far as mixing code compiled one way with code compiled the other way,
there is no problem, at least not more than on any other platform.
if we are heading for thread-safe libpq, I'd suggest, at least on UnixWare,
compiling
libpq with -D_REENTRANT, because of the macro vs. function issue, at least
for errno,
and possibly others.
LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Shridhar Daithankar writes:
On linux and freeBSD, that is not an issue.
FreeBSD has a separate libc_r. I don't think they would bother with that
if there was no significant difference.
I would really like to know for what platforms, locking mutex while selecting
from connection list or some such object, is going to be such a great
performance hit.
The performance hit isn't in the mutexes or specifically here or there.
But when you go into thread-safe/reentrant land, wild and wonderful things
happen, different on each platform. We could work out a specific solution
for each platform that fits well with the local custom and is most
convenient to users. But that is not maintainable. So we need to work
out some kind of generalization that works similarly on most platforms.
I'm still unsure what that should really be, but everything thread-safe by
default probably won't suffice.
--
Peter Eisentraut peter_e@gmx.net
Peter Eisentraut wrote:
A final note on the name of the configure option, --with-threads. First,
it does not control an external package but an internal feature, so it
should be --enable-. Secondly, it does not use threads, only enable
thread-safeness. So --enable-thread-safe might be a better name. Or if
you want to be more precise, --enable-thread-safe-client. The latter is
what MySQL uses, in case anyone cares about that.
Here is an applied patch to implement the rename you suggested, to
--enable-thread-safeness.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073Attachments:
/bjm/difftext/plainDownload+45-45
Shridhar Daithankar wrote:
I repeat what I have said earlier. If there are two libraries A using libecpg_r
and B, using libecpg, then program linking against both of them is going to
have tough time living with symbol conflicts.I suppose problem will be reproducible even under freeBSD if you try to create
a postgresql function in C which uses threads. Link the library against libc_r
and link postgresql against libc. It would run into problems.I am just stating my experiences.I might have missed solution to this problem.
But overall I like GNU libc approach of everything thread safe by default. If
thread performance is an issue, then it should be improved. Not worked around
with two libraries.
I thought glibc was the one to introduce libc_r in the first place ---
are they making libc thread-safe now?
What OS's are still using libc_r for threaded-ness? I never liked that
approach myself, and I resist adding it to our setup unless it is
required.
One problem now is that we don't have a way to create a separate libpq_r
for operating systems that use libc_r. We just create libpq and it is
thread-safe.
As for the configure flag, we still need it because we don't know the
flags required by all our supported OS's.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073Larry, please see configure.in and the template file and report the
changes needed for unixware and sco. See linux and freebsd for
examples.
---------------------------------------------------------------------------
Larry Rosenman wrote:
--On Wednesday, July 23, 2003 12:12:45 +0200 Peter Eisentraut
<peter_e@gmx.net> wrote:Larry Rosenman writes:
Please see my post to -hackers last week about -K[p]thread on UnixWare.
There are some issues there for us.The take-home message from those posts is that if you use non-thread-safe
code in threaded applications, bad things will happen. We knew that. ;-)
As far as mixing code compiled one way with code compiled the other way,
there is no problem, at least not more than on any other platform.if we are heading for thread-safe libpq, I'd suggest, at least on UnixWare,
compiling
libpq with -D_REENTRANT, because of the macro vs. function issue, at least
for errno,
and possibly others.LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073--On Wednesday, July 23, 2003 13:38:25 -0400 Bruce Momjian
<pgman@candle.pha.pa.us> wrote:
Larry, please see configure.in and the template file and report the
changes needed for unixware and sco. See linux and freebsd for
examples.
Can I deal with this after the beta tarball is up?
LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
Larry Rosenman wrote:
--On Wednesday, July 23, 2003 13:38:25 -0400 Bruce Momjian
<pgman@candle.pha.pa.us> wrote:Larry, please see configure.in and the template file and report the
changes needed for unixware and sco. See linux and freebsd for
examples.Can I deal with this after the beta tarball is up?
Sure. It can be fiddled with up until final release.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073On 23 Jul 2003 at 13:35, Bruce Momjian wrote:
I thought glibc was the one to introduce libc_r in the first place ---
are they making libc thread-safe now?
AFAIK, glibc plays all threadsafe in glibc2. Don't know prior to that. Of
course they do record if some of the routines are not thread safe but there is
no separate library as such.
by now, it is over all threadsafe. I don't have list of exceptions, but surely
they are pretty small, if any.
What OS's are still using libc_r for threaded-ness? I never liked that
approach myself, and I resist adding it to our setup unless it is
required.
FreeBSD and windows..:-) Two I know of..
As for the configure flag, we still need it because we don't know the
flags required by all our supported OS's.
Agreed.
Bye
Shridhar
--
"Waving away a cloud of smoke, I look up, and am blinded by a bright, whitelight. It's God. No, not Richard Stallman, or Linus Torvalds, but God. Ina booming voice, He says: "THIS IS A SIGN. USE LINUX, THE FREE UNIX SYSTEMFOR THE 386."(Matt Welsh)
Guys, take a look at what was done in libpq to make it
thread-safe... No locks! No overheaded - just using "proper" reentrant
functions...
If we have libpq_r then we're making a complete hash of it all - being
reentrant is good, even if you're not using threads!
Now, ecpg is another issue...
L.
Bruce Momjian writes:
Show quoted text
Shridhar Daithankar wrote:
I repeat what I have said earlier. If there are two libraries A using libecpg_r
and B, using libecpg, then program linking against both of them is going to
have tough time living with symbol conflicts.I suppose problem will be reproducible even under freeBSD if you try to create
a postgresql function in C which uses threads. Link the library against libc_r
and link postgresql against libc. It would run into problems.I am just stating my experiences.I might have missed solution to this problem.
But overall I like GNU libc approach of everything thread safe by default. If
thread performance is an issue, then it should be improved. Not worked around
with two libraries.I thought glibc was the one to introduce libc_r in the first place ---
are they making libc thread-safe now?What OS's are still using libc_r for threaded-ness? I never liked that
approach myself, and I resist adding it to our setup unless it is
required.One problem now is that we don't have a way to create a separate libpq_r
for operating systems that use libc_r. We just create libpq and it is
thread-safe.As for the configure flag, we still need it because we don't know the
flags required by all our supported OS's.
Lee Kindness wrote:
Guys, take a look at what was done in libpq to make it
thread-safe... No locks! No overheaded - just using "proper" reentrant
functions...If we have libpq_r then we're making a complete hash of it all - being
reentrant is good, even if you're not using threads!Now, ecpg is another issue...
I think the issue is that using a threaded library to link into libpq
could have locking stuff.
My guess is that if the OS has separate threaded libs, we have to mimic
that stuff.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073Shridhar Daithankar wrote:
On 23 Jul 2003 at 13:35, Bruce Momjian wrote:
I thought glibc was the one to introduce libc_r in the first place ---
are they making libc thread-safe now?AFAIK, glibc plays all threadsafe in glibc2. Don't know prior to that. Of
course they do record if some of the routines are not thread safe but there is
no separate library as such.by now, it is over all threadsafe. I don't have list of exceptions, but surely
they are pretty small, if any.What OS's are still using libc_r for threaded-ness? I never liked that
approach myself, and I resist adding it to our setup unless it is
required.FreeBSD and windows..:-) Two I know of..
The strange thing is that accoring to template/freebsd, libc_r was
_added_ in FreeBSD 5.0, while most OS are moving away from separate
threaded libs.
--
Bruce Momjian | http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073--On Thursday, July 24, 2003 08:52:37 -0400 Bruce Momjian
<pgman@candle.pha.pa.us> wrote:
Lee Kindness wrote:
Guys, take a look at what was done in libpq to make it
thread-safe... No locks! No overheaded - just using "proper" reentrant
functions...If we have libpq_r then we're making a complete hash of it all - being
reentrant is good, even if you're not using threads!Now, ecpg is another issue...
I think the issue is that using a threaded library to link into libpq
could have locking stuff.My guess is that if the OS has separate threaded libs, we have to mimic
that stuff.
possibly. On UnixWare, the original executable in the process needs to
have -K[p]thread
specified to get libthread included. I'll be adding -D_REENTRANT to the
libpq build for
UnixWare to make sure that libpq is threadsafe, but doesn't include the
thread library as
a pre-requisite per input from SCO.
BTW, Bruce, SCO OpenServer does NOT have threads.
LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749
On 24 Jul 2003 at 8:59, Bruce Momjian wrote:
Shridhar Daithankar wrote:
FreeBSD and windows..:-) Two I know of..
The strange thing is that accoring to template/freebsd, libc_r was
_added_ in FreeBSD 5.0, while most OS are moving away from separate
threaded libs.
Well, I have used libc_r for native freeBSD threads in 4.7 so that information
is certainly not correct..
Bye
Shridhar
--
Beam me up, Scotty! It ate my phaser!
Bruce Momjian writes:
Lee Kindness wrote:
Guys, take a look at what was done in libpq to make it
thread-safe... No locks! No overheaded - just using "proper" reentrant
functions...
If we have libpq_r then we're making a complete hash of it all - being
reentrant is good, even if you're not using threads!
Now, ecpg is another issue...I think the issue is that using a threaded library to link into libpq
could have locking stuff.My guess is that if the OS has separate threaded libs, we have to mimic
that stuff.
But there are NO thread primitives/calls in libpq - it's just a normal
shared library, it has has no thread voodoo! A threaded app/library
using libpq doesn't have to do anything special... libpq doesn't need
link the threads library (and it explicitly should not) - only define
_REENTRANT.
The reentrant calls libpq may be using are in the standard system
libraries (not the system thread library), the same libraries that the
app/libs will be linked against.
Now, libecpg on the otherhand does need mutex locks compiled in. Thus
if this is enabled then anyone who links against it also needs to link
in threads (unless the system is "intelligent" like Solaris and
includes empty stubs of the thread functions in libc).
However in this day and age the overhead of the locks are negligible
and it is more desirable to have one library...
L.
