thread_test.c problems

Started by Wesabout 22 years ago37 messageshackersgeneral
Jump to latest
#1Wes
wespvp@syntegra.com
hackersgeneral

[Sorry, I hit the wrong butt on and sent this earlier before I meant to]

When I try to run thread_test.c from the CVS tip, it hangs in an infinite
CPU loop on both linux (RedHat AS 3.0, gcc 3.2.3) and Mac OS X 10.3.3 (gcc
3.3). I've also tried down to gcc 2.96 on Mac OS X.

If I compile it with -g instead of -O2 on linux, it runs to completion and
gives me:

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no

On Mac OS X, it gives me:

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=yes

If I modify the two while loops

while (errno2_set == 0)

while (errno1_set == 0)

To execute a 'print' statement or a 'time(NULL)' instead of a null loop,
then it runs to completion with -O2.

If I run the 7.4.2 released version of thread_test.c, for both platforms I
get:

Add this to your template/$port file:

NEED_REENTRANT_FUNCS=yes

There seems to be two problems

1. It would appear the compiler is optimizing the while loop to be an
infinite loop. Other explanations?

2. The answers both can't be right... On OS X, the 7.4.2 version says
NEED_REENTRANT_FUNCS=yes, but with the tip version it says all are thread
safe.

On linux, the make run (putting 'time(NULL)' in the loop):

make
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -I../../../src/include -D_GNU_SOURCE -c -o
thread_test.o thread_test.c -MMD
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -L../../../src/port
-Wl,-rpath,/wdisk/wrp/opensrc/as30/local/pgsql7.4.2/lib -lpthread
thread_test.o -lz -lcrypt -lresolv -lnsl -ldl -lm -lbsd -lpgport -o
thread_test
[wrp@juniper thread]$ !.
./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no

#2joseph speigle
joe.speigle@jklh.us
In reply to: Wes (#1)
hackersgeneral
Re: thread_test.c problems

wespvp,

have you tried adding a volatile keyword to the

int volatile errno1_set = 0;
int volatile errno2_set = 0;

that should stop while-loop optimizing for cases where the variable is modified in a scope the compiler would be ignorant of, and precludes compiler while-loop optimization. That would then tell you if it was a while loop optimization problem.

As for me on RH 8.0

gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --host=i386-redhat-linux --with-system-zlib --enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)

sorry, they all run yes/no/no

On Sat, Apr 03, 2004 at 03:40:01PM -0600, wespvp@syntegra.com wrote:

[Sorry, I hit the wrong butt on and sent this earlier before I meant to]

When I try to run thread_test.c from the CVS tip, it hangs in an infinite
CPU loop on both linux (RedHat AS 3.0, gcc 3.2.3) and Mac OS X 10.3.3 (gcc
3.3). I've also tried down to gcc 2.96 on Mac OS X.

If I compile it with -g instead of -O2 on linux, it runs to completion and
gives me:

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no

On Mac OS X, it gives me:

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=yes

If I modify the two while loops

while (errno2_set == 0)

while (errno1_set == 0)

To execute a 'print' statement or a 'time(NULL)' instead of a null loop,
then it runs to completion with -O2.

If I run the 7.4.2 released version of thread_test.c, for both platforms I
get:

Add this to your template/$port file:

NEED_REENTRANT_FUNCS=yes

There seems to be two problems

1. It would appear the compiler is optimizing the while loop to be an
infinite loop. Other explanations?

2. The answers both can't be right... On OS X, the 7.4.2 version says
NEED_REENTRANT_FUNCS=yes, but with the tip version it says all are thread
safe.

On linux, the make run (putting 'time(NULL)' in the loop):

make
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -I../../../src/include -D_GNU_SOURCE -c -o
thread_test.o thread_test.c -MMD
gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
-Wmissing-declarations -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS -L../../../src/port
-Wl,-rpath,/wdisk/wrp/opensrc/as30/local/pgsql7.4.2/lib -lpthread
thread_test.o -lz -lcrypt -lresolv -lnsl -ldl -lm -lbsd -lpgport -o
thread_test
[wrp@juniper thread]$ !.
./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

--
joe speigle
www.sirfsup.com

#3Wes
wespvp@syntegra.com
In reply to: joseph speigle (#2)
hackersgeneral
Re: thread_test.c problems

On 4/4/04 12:33 AM, "joseph speigle" <joe.speigle@jklh.us> wrote:

have you tried adding a volatile keyword to the

int volatile errno1_set = 0;
int volatile errno2_set = 0;

that should stop while-loop optimizing for cases where the variable is
modified in a scope the compiler would be ignorant of, and precludes compiler
while-loop optimization. That would then tell you if it was a while loop
optimization problem.

That cures the problem on all systems. What needs to be done to get this
updated in the tip?

As for me on RH 8.0

gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --host=i386-redhat-linux --with-system-zlib
--enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)

sorry, they all run yes/no/no

I don't know what is different, but I've tried it on RH 7.2, RH AS 3.0, and
Mac OS X. All hang in an infinite loop with the tip version of
thread_test.c, and all work if I add 'volatile' to the definitions of
errno1_set and errno2_set.

There's still the question of which version of the program is giving the
right answers - the 7.4.2 version or the tip version?

RH 7.2:

[wrp@cherry thread]$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/2.96/specs
gcc version 2.96 20000731 (Red Hat Linux 7.2 2.96-112.7.2)

RH AS 3.0:

$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.3/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --with-system-zlib --enable-__cxa_atexit
--host=i386-redhat-linux
Thread model: posix
gcc version 3.2.3 20030502 (Red Hat Linux 3.2.3-24)

Mac OS X:

% gcc -v
Reading specs from /usr/libexec/gcc/darwin/ppc/3.3/specs
Thread model: posix
gcc version 3.3 20030304 (Apple Computer, Inc. build 1495)

Wes

#4Bruce Momjian
bruce@momjian.us
In reply to: Wes (#3)
hackersgeneral
Re: thread_test.c problems

wespvp@syntegra.com wrote:

On 4/4/04 12:33 AM, "joseph speigle" <joe.speigle@jklh.us> wrote:

have you tried adding a volatile keyword to the

int volatile errno1_set = 0;
int volatile errno2_set = 0;

that should stop while-loop optimizing for cases where the variable is
modified in a scope the compiler would be ignorant of, and precludes compiler
while-loop optimization. That would then tell you if it was a while loop
optimization problem.

That cures the problem on all systems. What needs to be done to get this
updated in the tip?

As for me on RH 8.0

gcc -v
Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2/specs
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
--infodir=/usr/share/info --enable-shared --enable-threads=posix
--disable-checking --host=i386-redhat-linux --with-system-zlib
--enable-__cxa_atexit
Thread model: posix
gcc version 3.2 20020903 (Red Hat Linux 8.0 3.2-7)

sorry, they all run yes/no/no

I don't know what is different, but I've tried it on RH 7.2, RH AS 3.0, and
Mac OS X. All hang in an infinite loop with the tip version of
thread_test.c, and all work if I add 'volatile' to the definitions of
errno1_set and errno2_set.

There's still the question of which version of the program is giving the
right answers - the 7.4.2 version or the tip version?

I have applied the following patch to CVS head which does a getpid() in
the loop, rather than nothing. getpid() should force a system call,
which will make it more likely for the other thread to get CPU time and
complete its tests.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/bjm/difftext/plainDownload+12-12
#5Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#4)
hackersgeneral
Re: thread_test.c problems

I have applied the following patch to CVS head which does a getpid() in
the loop, rather than nothing. getpid() should force a system call,
which will make it more likely for the other thread to get CPU time and
complete its tests.

Works for me...

However, there seems to be a reliability problem with the results:

[g4:src/tools/thread] wp% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=no

[g4:src/tools/thread] wp% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=yes

It returns mostly 'yes' for all three, but sometimes returns 'no' for the
last one. I get the same conflicting results whether I run with one or two
CPU's enabled.

Similarly, the 7.4.2 release is showing:

% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

All your non-*_r functions are thread-safe.
Add this to your template/$port file:

NEED_REENTRANT_FUNCS=no

% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Your gethostbyname() is _not_ thread-safe
Not all non-*_r functions are thread-safe.
Add this to your template/$port file:

NEED_REENTRANT_FUNCS=yes

I assume the safe bet is GETHOSTBYNAME_THREADSAFE=no for the tip and
NEED_REENTRANT_FUNCS=yes for 7.4.2? Can I rely on the other two being
'yes', or should they be set to 'no' also?

Based on the linux template and the output of thread_test, I have set up the
darwin template and compiled the CVS tip code on Mac OS X 10.3.3 with:

# Apple's cpp-precomp seems a tad broken, so don't use it
# (Note: on OS X before 10.2, you might need -traditional-cpp instead)
#CC="$CC -no-cpp-precomp"

# Select appropriate semaphore support
USE_NAMED_POSIX_SEMAPHORES=1

# tools/thread/thread_test must be run
THREAD_SUPPORT=yes
THREAD_CPPFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
THREAD_LIBS="-lpthread"

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=no

#NEED_REENTRANT_FUNCS=yes

I commented out the 'CC="$CC -no-cpp-precomp"' line and did not get any
compile errors. Mac OS X 10.3.3 uses GCC 3.3. I know something changed
between Mac OS X 10.2 and 10.3 regarding precomp (they also upgraded from
gcc 2.96 to 3.3). It seems to compile ok on 10.3 regardless of the setting
of this parameter, so unless it is important to have it enabled when
possible, it is probably best to leave it off for backwards compatibility?

I also added all of the thread lines starting with the comment '#
tools/thread/thread_test must be run' based on the output of thread_test.
The released PostgreSQL 7.4.2 says I should use 'NEED_REENTRANT_FUNCS=yes'.

Wes

#6Bruce Momjian
bruce@momjian.us
In reply to: Wes (#5)
hackersgeneral
Re: thread_test.c problems

wespvp@syntegra.com wrote:

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=no

[g4:src/tools/thread] wp% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=yes

It returns mostly 'yes' for all three, but sometimes returns 'no' for the
last one. I get the same conflicting results whether I run with one or two
CPU's enabled.

OK, I know the cause of this. The problem is that sometimes hostnames
don't resolve, and the bigger problem is that it requires an internet
connection to run the tests. The attached patch tests for 'localhost'
and your local hostname, so it should work reliably.

Similarly, the 7.4.2 release is showing:

% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

All your non-*_r functions are thread-safe.
Add this to your template/$port file:

NEED_REENTRANT_FUNCS=no

% ./thread_test
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Your gethostbyname() is _not_ thread-safe
Not all non-*_r functions are thread-safe.
Add this to your template/$port file:

NEED_REENTRANT_FUNCS=yes

I assume the safe bet is GETHOSTBYNAME_THREADSAFE=no for the tip and
NEED_REENTRANT_FUNCS=yes for 7.4.2? Can I rely on the other two being
'yes', or should they be set to 'no' also?

Yes, that is the safe bet on 7.4.X. 7.5 will use individual entries for
each function, and in fact we might have this all merged into configure
by then anyway.

Based on the linux template and the output of thread_test, I have set up the
darwin template and compiled the CVS tip code on Mac OS X 10.3.3 with:

# Apple's cpp-precomp seems a tad broken, so don't use it
# (Note: on OS X before 10.2, you might need -traditional-cpp instead)
#CC="$CC -no-cpp-precomp"

# Select appropriate semaphore support
USE_NAMED_POSIX_SEMAPHORES=1

# tools/thread/thread_test must be run
THREAD_SUPPORT=yes
THREAD_CPPFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
THREAD_LIBS="-lpthread"

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=no

#NEED_REENTRANT_FUNCS=yes

I commented out the 'CC="$CC -no-cpp-precomp"' line and did not get any
compile errors. Mac OS X 10.3.3 uses GCC 3.3. I know something changed
between Mac OS X 10.2 and 10.3 regarding precomp (they also upgraded from
gcc 2.96 to 3.3). It seems to compile ok on 10.3 regardless of the setting
of this parameter, so unless it is important to have it enabled when
possible, it is probably best to leave it off for backwards compatibility?

I also added all of the thread lines starting with the comment '#
tools/thread/thread_test must be run' based on the output of thread_test.
The released PostgreSQL 7.4.2 says I should use 'NEED_REENTRANT_FUNCS=yes'.

Yes, in 7.4.X, you only need that single entry.

Thanks for the testing.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/bjm/difftext/plainDownload+18-10
#7Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#4)
hackersgeneral
Re: thread_test.c problems

Bruce Momjian <pgman@candle.pha.pa.us> writes:

wespvp@syntegra.com wrote:
have you tried adding a volatile keyword to the
int volatile errno1_set = 0;
int volatile errno2_set = 0;

I have applied the following patch to CVS head which does a getpid() in
the loop, rather than nothing.

That does not cure the bug identified by wespvp. It may make it a
little less likely that the compiler will choose to optimize out the
loop test, but only adding "volatile" really fixes the problem in
a language-standard-compliant way.

regards, tom lane

#8Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#7)
hackersgeneral
Re: thread_test.c problems

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

wespvp@syntegra.com wrote:
have you tried adding a volatile keyword to the
int volatile errno1_set = 0;
int volatile errno2_set = 0;

I have applied the following patch to CVS head which does a getpid() in
the loop, rather than nothing.

That does not cure the bug identified by wespvp. It may make it a
little less likely that the compiler will choose to optimize out the
loop test, but only adding "volatile" really fixes the problem in
a language-standard-compliant way.

Thanks, 'volatile' added to the thread-specific errno variables:

volatile int errno1_set = 0;
volatile int errno2_set = 0;

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#9Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#8)
hackersgeneral
Re: thread_test.c problems

wespvp@syntegra.com wrote:

On 4/4/04 7:28 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, I know the cause of this. The problem is that sometimes hostnames
don't resolve, and the bigger problem is that it requires an internet
connection to run the tests. The attached patch tests for 'localhost'
and your local hostname, so it should work reliably.

I did a cvs update, make, then rebuilt and tested thread_test. I ran it
about 120 times and got "GETHOSTBYNAME_THREADSAFE=yes" almost all of the
time. I got "GETHOSTBYNAME_THREADSAFE=no" three times. The other two were
always "yes".

Yep, I can reproduce this on BSD/OS too, so it must be something wrong
with my program. I see:

$ for X in `jot 1000`; do thread_test |grep -10 '=no' && echo $X;
done|less
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no
919

I bet the problem is that I am accessing thread-specific pointers after
the thread exits. It is possible thread 1 completes before thread2 gets
to the getpwuid and gethostbyname tests, and then reuses the
thread-specific pointer. Let me work on a patch and email it to you in
a few minutes.

I did verify that both the "volatile" and "localhost" changes were in there.

Dumb question... Why would you not always use the _r functions if they
exist?

Yes, we do use *_r functions in 7.5 if they exist, but in 7.4.X, I think
we use the non-R if we can, though we actually just use getaddrinfo() in
7.4.X if it exists. Basically, the threading tests are still in flux,
as you can see, in 7.4.X. It works, but it isn't 100% configure perfect
yet.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#10Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#6)
hackersgeneral
Re: thread_test.c problems

On 4/4/04 7:28 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, I know the cause of this. The problem is that sometimes hostnames
don't resolve, and the bigger problem is that it requires an internet
connection to run the tests. The attached patch tests for 'localhost'
and your local hostname, so it should work reliably.

I did a cvs update, make, then rebuilt and tested thread_test. I ran it
about 120 times and got "GETHOSTBYNAME_THREADSAFE=yes" almost all of the
time. I got "GETHOSTBYNAME_THREADSAFE=no" three times. The other two were
always "yes".

I did verify that both the "volatile" and "localhost" changes were in there.

Dumb question... Why would you not always use the _r functions if they
exist?

Wes

#11Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#9)
hackersgeneral
Re: thread_test.c problems

Bruce Momjian wrote:

I did a cvs update, make, then rebuilt and tested thread_test. I ran it
about 120 times and got "GETHOSTBYNAME_THREADSAFE=yes" almost all of the
time. I got "GETHOSTBYNAME_THREADSAFE=no" three times. The other two were
always "yes".

Yep, I can reproduce this on BSD/OS too, so it must be something wrong
with my program. I see:

$ for X in `jot 1000`; do thread_test |grep -10 '=no' && echo $X;
done|less
Make sure you have added any needed 'THREAD_CPPFLAGS' and 'THREAD_LIBS'
defines to your template/$port file before compiling this program.

Add this to your template/$port file:

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=no
GETHOSTBYNAME_THREADSAFE=no
919

I bet the problem is that I am accessing thread-specific pointers after
the thread exits. It is possible thread 1 completes before thread2 gets
to the getpwuid and gethostbyname tests, and then reuses the
thread-specific pointer. Let me work on a patch and email it to you in
a few minutes.

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/bjm/difftext/plainDownload+42-22
#12Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#11)
hackersgeneral
Re: thread_test.c problems

wespvp@syntegra.com wrote:

On 4/4/04 11:43 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Please grab CVS. I added sched_yield() too.

Dumb question... Why would you not always use the _r functions if they
exist?

Yes, we do use *_r functions in 7.5 if they exist, but in 7.4.X, I think
we use the non-R if we can, though we actually just use getaddrinfo() in
7.4.X if it exists. Basically, the threading tests are still in flux,
as you can see, in 7.4.X. It works, but it isn't 100% configure perfect
yet.

I'm still not clear on this... The thread_test program checks to see if the
non-r functions are thread safe. If so, it directs you to set xxxx=yes in
the template file - I assume that causes the non-r function to be used. If
they are not thread safe, it directs you to use xxxx=no - which I assume
causes the *_r functions to be used. Why would you not *always* use the _r
functions if they exist, and only check for thread safety if the _r
functions do not exist?

Or, am I misunderstanding how the xxx=yes is used?

In 7.4.X, the thought was that if the native function are already
thread-safe, why bother calling the *_r functions, but in 7.5, we
decided it was better to use the *_r functions --- again, 7.4.X has
threads working, but the configure issues were in flux as we learned how
each platform handled threading.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#13Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#11)
hackersgeneral
Re: thread_test.c problems

On 4/4/04 11:43 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Dumb question... Why would you not always use the _r functions if they
exist?

Yes, we do use *_r functions in 7.5 if they exist, but in 7.4.X, I think
we use the non-R if we can, though we actually just use getaddrinfo() in
7.4.X if it exists. Basically, the threading tests are still in flux,
as you can see, in 7.4.X. It works, but it isn't 100% configure perfect
yet.

I'm still not clear on this... The thread_test program checks to see if the
non-r functions are thread safe. If so, it directs you to set xxxx=yes in
the template file - I assume that causes the non-r function to be used. If
they are not thread safe, it directs you to use xxxx=no - which I assume
causes the *_r functions to be used. Why would you not *always* use the _r
functions if they exist, and only check for thread safety if the _r
functions do not exist?

Or, am I misunderstanding how the xxx=yes is used?

Wes

#14Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#12)
hackersgeneral
Re: thread_test.c problems

Bruce Momjian <pgman@candle.pha.pa.us> writes:

wespvp@syntegra.com wrote:

On 4/4/04 11:43 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Please grab CVS. I added sched_yield() too.

Make sure to test this on linux 2.6. They changed sched_yield() to not cause
context switches in a lot of cases where it did previously. You might need a
usleep() there instead or in addition.

--
greg

#15Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#14)
hackersgeneral
Re: thread_test.c problems

Greg Stark wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

wespvp@syntegra.com wrote:

On 4/4/04 11:43 PM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Please grab CVS. I added sched_yield() too.

Make sure to test this on linux 2.6. They changed sched_yield() to not cause
context switches in a lot of cases where it did previously. You might need a
usleep() there instead or in addition.

Usleep is probably more unportable than sched_yield on platforms that
support threads. I actually don't need sched_yield to context switch,
only to allow the other threads to complete.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#16Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#12)
hackersgeneral
Re: thread_test.c problems

On 4/5/04 8:31 AM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Please grab CVS. I added sched_yield() too.

Ok, I did a cvs update and reran the test. It now consistently returns
'YES' to all three questions. Thanks.

What needs to be done to get the standard Darwin template file updated to
enable threading?

Wes

#17Bruce Momjian
bruce@momjian.us
In reply to: Wes (#16)
hackersgeneral
Re: thread_test.c problems

wespvp@syntegra.com wrote:

On 4/5/04 8:31 AM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

OK, new patch applied that causes all threads to wait until the parent
checks their thread-specific pointers. I ran 1000 tests and all passed.
Hopefully it will good for you too.

I'll try to give it a test tonight.

Please grab CVS. I added sched_yield() too.

Ok, I did a cvs update and reran the test. It now consistently returns
'YES' to all three questions. Thanks.

What needs to be done to get the standard Darwin template file updated to
enable threading?

Oh, that is a problem because other linux's show 'no' for the last two.

My plan is to have 'configure' run this program as part of its
operation, and Jan has some thread configure scripts, and this will all
be automated in 7.5 so no one will have to run the program manually.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#18Tom Lane
tgl@sss.pgh.pa.us
In reply to: Bruce Momjian (#17)
hackersgeneral
Re: thread_test.c problems

Bruce Momjian <pgman@candle.pha.pa.us> writes:

My plan is to have 'configure' run this program as part of its
operation,

Peter's not going to be happy with you ;-) Runtime configure tests are
not part of the Grand Plan --- ideally configure should only have to do
compile and link tests.

regards, tom lane

#19Bruce Momjian
bruce@momjian.us
In reply to: Tom Lane (#18)
hackersgeneral
Re: thread_test.c problems

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

My plan is to have 'configure' run this program as part of its
operation,

Peter's not going to be happy with you ;-) Runtime configure tests are
not part of the Grand Plan --- ideally configure should only have to do
compile and link tests.

Yep, I am sure he isn't, but looking at platforms that have different
thread-safety capabilities, I can't see a way around it. We could
argue that the thread test is testing the thread library, not the actual
computer it is running in. For example, it isn't do a 'df'.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#20Wes Palmer
Wesley.R.Palmer@syntegra.com
In reply to: Bruce Momjian (#19)
general
Re: thread_test.c problems

On 4/7/04 9:22 AM, "Bruce Momjian" <pgman@candle.pha.pa.us> wrote:

Are any special thread flags needed? What version of Darwin are you
using?

I am running Mac OS X 10.3.3. Uname -a shows that as

Darwin Kernel Version 7.3.0

I took the flags from the linux version. It appears to work ok - at least
it compiles and runs ok. I'm not sure how to verify threading is working
correctly... What I have is:

# Apple's cpp-precomp seems a tad broken, so don't use it
# (Note: on OS X before 10.2, you might need -traditional-cpp instead)
CC="$CC -no-cpp-precomp"

# Select appropriate semaphore support
USE_NAMED_POSIX_SEMAPHORES=1

# tools/thread/thread_test must be run
THREAD_SUPPORT=yes
THREAD_CPPFLAGS="-D_REENTRANT -D_THREAD_SAFE -D_POSIX_PTHREAD_SEMANTICS"
THREAD_LIBS="-lpthread"

STRERROR_THREADSAFE=yes
GETPWUID_THREADSAFE=yes
GETHOSTBYNAME_THREADSAFE=yes

#21Wes
wespvp@syntegra.com
In reply to: Wes Palmer (#20)
general
#22Bruce Momjian
bruce@momjian.us
In reply to: Wes Palmer (#20)
general
#23Bruce Momjian
bruce@momjian.us
In reply to: Wes (#21)
general
#24Bruce Momjian
bruce@momjian.us
In reply to: Wes (#16)
hackersgeneral
#25Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#24)
hackersgeneral
#26Bruce Momjian
bruce@momjian.us
In reply to: Wes (#25)
hackersgeneral
#27Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#26)
hackersgeneral
#28Bruce Momjian
bruce@momjian.us
In reply to: Wes (#27)
hackersgeneral
#29Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#28)
hackersgeneral
#30Bruce Momjian
bruce@momjian.us
In reply to: Wes (#29)
hackersgeneral
#31Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#30)
hackersgeneral
#32Wes Palmer
Wesley.R.Palmer@syntegra.com
In reply to: Bruce Momjian (#30)
hackersgeneral
#33Bruce Momjian
bruce@momjian.us
In reply to: Wes Palmer (#32)
hackersgeneral
#34Bruce Momjian
bruce@momjian.us
In reply to: Bruce Momjian (#33)
hackersgeneral
#35Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#34)
hackersgeneral
#36Wes
wespvp@syntegra.com
In reply to: Bruce Momjian (#34)
hackersgeneral
#37Bruce Momjian
bruce@momjian.us
In reply to: Wes (#36)
hackersgeneral