V8 Beta 5 on AIX

Brad Nicholson wrote:

I tried compliling v8 beta 5 (grabbed from cvs on Friday) on AIX 5.1.
It fails during the make

make[4]: Entering directory
`/opt/OXRS/Sources/pgsql-HEAD/src/interfaces/libpq'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory
`/opt/OXRS/Sources/pgsql-HEAD/src/interfaces/libpq'
make[4]: Entering directory `/opt/OXRS/Sources/pgsql-HEAD/src/port'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/port'
gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels
-fno-strict-aliasing -g pg_ctl.o -L../../../src/interfaces/libpq -lpq
-L../../../src/port -L/opt/OXRS/supporting-packages/readline-4.2/lib
-L/opt/OXRS/supporting-packages/zlib-1.1.4 -lpgport -lz -lreadline
-lcurses -lPW -lld -lnsl -ldl -lm -o pg_ctl
ld: 0711-317 ERROR: Undefined symbol: .pthread_sigmask
ld: 0711-317 ERROR: Undefined symbol: .sigwait
ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_lock
ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_unlock
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
collect2: ld returned 8 exit status
make[3]: *** [pg_ctl] Error 1
make[3]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/bin/pg_ctl'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/bin'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src'
make: *** [all] Error 2

OK, I assume you used --enable-thread-safety in configure. This should
have added some PTHREAD link flags to your libpq build, and those
settings should have followed the libpq library into your pg_ctl link
line.

Would you look in your Makefile.global for PTHREAD_* settings and report
those. The second question is why saying those libraries are needed by
libpq is not passing down to uses of libpq, like in pg_ctl.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

Brad Nicholson wrote:

I tried compliling v8 beta 5 (grabbed from cvs on Friday) on AIX 5.1.
It fails during the make

make[4]: Entering directory
`/opt/OXRS/Sources/pgsql-HEAD/src/interfaces/libpq'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory
`/opt/OXRS/Sources/pgsql-HEAD/src/interfaces/libpq'
make[4]: Entering directory `/opt/OXRS/Sources/pgsql-HEAD/src/port'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/port'
gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels
-fno-strict-aliasing -g pg_ctl.o -L../../../src/interfaces/libpq -lpq
-L../../../src/port -L/opt/OXRS/supporting-packages/readline-4.2/lib
-L/opt/OXRS/supporting-packages/zlib-1.1.4 -lpgport -lz -lreadline
-lcurses -lPW -lld -lnsl -ldl -lm -o pg_ctl
ld: 0711-317 ERROR: Undefined symbol: .pthread_sigmask
ld: 0711-317 ERROR: Undefined symbol: .sigwait
ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_lock
ld: 0711-317 ERROR: Undefined symbol: .pthread_mutex_unlock
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
collect2: ld returned 8 exit status
make[3]: *** [pg_ctl] Error 1
make[3]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/bin/pg_ctl'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src/bin'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/opt/OXRS/Sources/pgsql-HEAD/src'
make: *** [all] Error 2

OK, I assume you used --enable-thread-safety in configure.

Correct.

This should
have added some PTHREAD link flags to your libpq build, and those
settings should have followed the libpq library into your pg_ctl link
line.

Would you look in your Makefile.global for PTHREAD_* settings and report
those. The second question is why saying those libraries are needed by
libpq is not passing down to uses of libpq, like in pg_ctl.

PTHREAD_CFLAGS = -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS
PTHREAD_LIBS = -lpthread -lpthreads

Brad.

Brad Nicholson wrote:

OK, I assume you used --enable-thread-safety in configure.

Correct.

This should
have added some PTHREAD link flags to your libpq build, and those
settings should have followed the libpq library into your pg_ctl link
line.

Would you look in your Makefile.global for PTHREAD_* settings and report
those. The second question is why saying those libraries are needed by
libpq is not passing down to uses of libpq, like in pg_ctl.

PTHREAD_CFLAGS = -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS
PTHREAD_LIBS = -lpthread -lpthreads

OK, great. Can I see the link line that creates libpq? It should contain
the PTHREAD_LIBS flags.

If it does, the next question is why AIX doesn't pass those flags.

Oh, I think I see the cause. I now remember from
config/acx_pthread.m4:

# More AIX lossage: must compile with cc_r
AC_CHECK_PROG(PTHREAD_CC, cc_r, cc_r, ${CC})

Seems AIX wants a special _compiler_ to be used whenever threading is
involved, even by a linked-in library. I didn't implement using
PTHREAD_CC because there was just too much code disruption for threading
on one platform. I wonder if we should just disable threading on AIX.

Let me ask --- if you change the CC line in Makefile.global to cc_r,
does everything build OK? That might be a clean solution because the
change could be made in one place. Of course this would mean the
backend would also be compiled using cc_r and I have no idea of the
effect.

Of course, the idea that any use of libpq has to use cc_r is going to
make building things complex without some adjustments.

Also, what version of AIX are you using? Are other AIX folks having
thread build problems?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Mon, Dec 06, 2004 at 12:53:52PM -0500, Bruce Momjian wrote:

Brad Nicholson wrote:

OK, I assume you used --enable-thread-safety in configure.

Correct.

This should
have added some PTHREAD link flags to your libpq build, and those
settings should have followed the libpq library into your pg_ctl link
line.

Would you look in your Makefile.global for PTHREAD_* settings and report
those. The second question is why saying those libraries are needed by
libpq is not passing down to uses of libpq, like in pg_ctl.

PTHREAD_CFLAGS = -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS
PTHREAD_LIBS = -lpthread -lpthreads

OK, great. Can I see the link line that creates libpq? It should contain
the PTHREAD_LIBS flags.

If it does, the next question is why AIX doesn't pass those flags.

Oh, I think I see the cause. I now remember from
config/acx_pthread.m4:

# More AIX lossage: must compile with cc_r
AC_CHECK_PROG(PTHREAD_CC, cc_r, cc_r, ${CC})

Seems AIX wants a special _compiler_ to be used whenever threading is
involved, even by a linked-in library. I didn't implement using
PTHREAD_CC because there was just too much code disruption for threading
on one platform. I wonder if we should just disable threading on AIX.

Let me ask --- if you change the CC line in Makefile.global to cc_r,
does everything build OK? That might be a clean solution because the
change could be made in one place. Of course this would mean the
backend would also be compiled using cc_r and I have no idea of the
effect.

Of course, the idea that any use of libpq has to use cc_r is going to
make building things complex without some adjustments.

Also, what version of AIX are you using? Are other AIX folks having
thread build problems?

We have until very recently supporting a number of applications
requiring threading on AIX 3/4/5. They always required much more
understanding of the entire compile/link/run cycle than any other
platform we use. Changing the CC line to cc_r works fine with the
only problem being the cascaded use of cc_r into any application
that links against libpq. Even with these complications, I think
we should still allow threading on AIX. Anyone who builds software
on AIX already knows how to manage these issues.

Ken

Brad Nicholson wrote:

I tried compliling v8 beta 5 (grabbed from cvs on Friday) on AIX 5.1.

I see now you are running AIX 5.1. Is that a fairly modern/popular
version of AIX?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

Brad Nicholson wrote:

OK, I assume you used --enable-thread-safety in configure.

Correct.

This should
have added some PTHREAD link flags to your libpq build, and those
settings should have followed the libpq library into your pg_ctl link
line.

Would you look in your Makefile.global for PTHREAD_* settings and report
those. The second question is why saying those libraries are needed by
libpq is not passing down to uses of libpq, like in pg_ctl.

PTHREAD_CFLAGS = -pthread -D_REENTRANT -D_THREAD_SAFE
-D_POSIX_PTHREAD_SEMANTICS
PTHREAD_LIBS = -lpthread -lpthreads

OK, great. Can I see the link line that creates libpq? It should contain
the PTHREAD_LIBS flags.

I've attached the Makefile.global from the system, in case it's of use.

If it does, the next question is why AIX doesn't pass those flags.

Oh, I think I see the cause. I now remember from
config/acx_pthread.m4:

# More AIX lossage: must compile with cc_r
AC_CHECK_PROG(PTHREAD_CC, cc_r, cc_r, ${CC})

Seems AIX wants a special _compiler_ to be used whenever threading is
involved, even by a linked-in library. I didn't implement using
PTHREAD_CC because there was just too much code disruption for threading
on one platform. I wonder if we should just disable threading on AIX.

I believe that this will result in Slony not working on AIX.

Let me ask --- if you change the CC line in Makefile.global to cc_r,
does everything build OK? That might be a clean solution because the
change could be made in one place. Of course this would mean the
backend would also be compiled using cc_r and I have no idea of the
effect.

Not an option, we don't have cc_r on the server.

Of course, the idea that any use of libpq has to use cc_r is going to
make building things complex without some adjustments.

Also, what version of AIX are you using? Are other AIX folks having
thread build problems?

5.1

Brad.

On Mon, Dec 06, 2004 at 01:07:11PM -0500, Bruce Momjian wrote:

I see now you are running AIX 5.1. Is that a fairly modern/popular
version of AIX?

To the extent AIX is popular :) 5.1 is one release behind the very
latest.

A

--
Andrew Sullivan | ajs@crankycanuck.ca
The plural of anecdote is not data.
--Roger Brinner

On Mon, 2004-12-06 at 21:59, Andrew Sullivan wrote:

On Mon, Dec 06, 2004 at 01:07:11PM -0500, Bruce Momjian wrote:

I see now you are running AIX 5.1. Is that a fairly modern/popular
version of AIX?

To the extent AIX is popular :) 5.1 is one release behind the very
latest.

AIX 5.3 is out now...

AIX 5.1 is still fully supported though...

--
Best Regards, Simon Riggs

Simon Riggs wrote:

On Mon, 2004-12-06 at 21:59, Andrew Sullivan wrote:

On Mon, Dec 06, 2004 at 01:07:11PM -0500, Bruce Momjian wrote:

I see now you are running AIX 5.1. Is that a fairly modern/popular
version of AIX?

To the extent AIX is popular :) 5.1 is one release behind the very
latest.

AIX 5.3 is out now...

AIX 5.1 is still fully supported though...

OK, so does someone want to suggest why a library used to link libpq
would also be needed to link binaries that use libpq? And with no cc_r
it seems I have no idea how to get this working.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

OK, so does someone want to suggest why a library used to link libpq
would also be needed to link binaries that use libpq?

No doubt because it's one of those platforms where shared libraries
don't carry their own dependency information. AFAICS, PTHREAD_LIBS is
not propagated into the link lines of pg_ctl or any of the other
programs that use libpq, so the only place where the linker would find
out about it is if the info is embedded in libpq.so.

regards, tom lane

Bruce Momjian wrote:

OK, so does someone want to suggest why a library used to link libpq
would also be needed to link binaries that use libpq? And with no
cc_r it seems I have no idea how to get this working.

We didn't get this working for 7.4, for reasons that we are again
becoming aware of, so I don't think it's justifiable to try to fix it
for 8.0 at this point.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Bruce Momjian wrote:

OK, so does someone want to suggest why a library used to link libpq
would also be needed to link binaries that use libpq? And with no
cc_r it seems I have no idea how to get this working.

We didn't get this working for 7.4, for reasons that we are again
becoming aware of, so I don't think it's justifiable to try to fix it
for 8.0 at this point.

Yea, we needed this problem report during beta, not RC.

I added this to Makefile.unixware long ago:

# Unixware needs threads for everything that uses libpq
CFLAGS += $(PTHREAD_CFLAGS)

I said if we found another platform that had a similar limitation we
would fix it more thoroughly, but I needed to fix it earlier to get into
8.0.

However, one thing we can do is to try this in Makefile.aix:

# AIX needs threads for everything that uses libpq
LIBS += $(PTHREAD_LIBS)

That is going to enable thread libs for all linking including the
backend, but it might work.

Unixware found that doing this for the backend brought out threading OS
bugs and it was useless but AIX might be better.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Kenneth Marshall wrote:

Let me ask --- if you change the CC line in Makefile.global to cc_r,
does everything build OK? That might be a clean solution because the
change could be made in one place. Of course this would mean the
backend would also be compiled using cc_r and I have no idea of the
effect.

Of course, the idea that any use of libpq has to use cc_r is going to
make building things complex without some adjustments.

Also, what version of AIX are you using? Are other AIX folks having
thread build problems?

We have until very recently supporting a number of applications
requiring threading on AIX 3/4/5. They always required much more
understanding of the entire compile/link/run cycle than any other
platform we use. Changing the CC line to cc_r works fine with the
only problem being the cascaded use of cc_r into any application
that links against libpq. Even with these complications, I think
we should still allow threading on AIX. Anyone who builds software
on AIX already knows how to manage these issues.

OK, but why does the AIX 5.1 version they have not have cc_r?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

However, one thing we can do is to try this in Makefile.aix:
# AIX needs threads for everything that uses libpq
LIBS += $(PTHREAD_LIBS)
That is going to enable thread libs for all linking including the
backend, but it might work.

That is certainly wrong. The correct thing is to add PTHREAD_LIBS to
all and only those links that include libpq. I suspect that the cc_r
business is a red herring and the real problem is just that you forgot
to include the libraries --- if so, AIX is probably not the only
platform that will break.

regards, tom lane

OK, so does someone want to suggest why a library used to link libpq
would also be needed to link binaries that use libpq? And with no cc_r
it seems I have no idea how to get this working.

AIX does not pull in any libraries that a shared lib depends upon, not even libc.
You only specify those on the .so link line so the linker can check what
is unresolved.

If he does not have cc_r, then he eighter uses gcc or a newer compiler,
that does not need cc_r any more ???

I would generally use cc_r if it exists or gcc for the threaded build.
The performance difference between cc and cc_r should not be noticeable.

Andreas

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

However, one thing we can do is to try this in Makefile.aix:
# AIX needs threads for everything that uses libpq
LIBS += $(PTHREAD_LIBS)
That is going to enable thread libs for all linking including the
backend, but it might work.

That is certainly wrong. The correct thing is to add PTHREAD_LIBS to
all and only those links that include libpq. I suspect that the cc_r
business is a red herring and the real problem is just that you forgot
to include the libraries --- if so, AIX is probably not the only
platform that will break.

The discussion at the time was that we didn't want to add a
libpq-specific CC just for Unixware, which we considered broken for not
having libpq dependency information in the library.

Now that we have two platforms that need this we might need to rethink
this, and think of a shortcut for 8.0 like we are already using for
Unixware.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Mon, Dec 06, 2004 at 10:17:53PM +0000, Simon Riggs wrote:

To the extent AIX is popular :) 5.1 is one release behind the very
latest.

AIX 5.3 is out now...

AIX 5.1 is still fully supported though...

Oops, sorry, I meant "one behind the very latest supported by our
provider". For reasons I don't quite understand, IBM's own hosting
facilities seem to lag on the AIX releases.

A

--
Andrew Sullivan | ajs@crankycanuck.ca
I remember when computers were frustrating because they *did* exactly what
you told them to. That actually seems sort of quaint now.
--J.D. Baldwin

On Mon, Dec 06, 2004 at 08:07:00PM -0500, Bruce Momjian wrote:

Yea, we needed this problem report during beta, not RC.

For the record, there _was_ such a problem report, for b3:

http://archives.postgresql.org/pgsql-hackers/2004-10/msg00046.php

A

--
Andrew Sullivan | ajs@crankycanuck.ca
A certain description of men are for getting out of debt, yet are
against all taxes for raising money to pay it off.
--Alexander Hamilton

However, one thing we can do is to try this in Makefile.aix:
# AIX needs threads for everything that uses libpq
LIBS += $(PTHREAD_LIBS)
That is going to enable thread libs for all linking including the
backend, but it might work.

That is certainly wrong. The correct thing is to add PTHREAD_LIBS to
all and only those links that include libpq. I suspect that the cc_r

You mean like attached patch ? That is safe and can be applied.
My machine has problems with two configure tests with --enable-thread-safety.

1. the snprintf long long int check that uses int64 as variable name which is
a typedef in sys/inttypes.h
2. whether strerror_r returns int check fails with a redefine of named function
from string.h

No idea how to fix those, but workaround is easy :-(

-lpthread can be removed from PTHREAD_LIBS, the lib is a compat symlink to
-lpthreads, I guess it does not hurt eighter.

Andreas

pgman@candle.pha.pa.us (Bruce Momjian) writes:

Kenneth Marshall wrote:

Let me ask --- if you change the CC line in Makefile.global to cc_r,
does everything build OK? That might be a clean solution because the
change could be made in one place. Of course this would mean the
backend would also be compiled using cc_r and I have no idea of the
effect.

Of course, the idea that any use of libpq has to use cc_r is going to
make building things complex without some adjustments.

Also, what version of AIX are you using? Are other AIX folks having
thread build problems?

We have until very recently supporting a number of applications
requiring threading on AIX 3/4/5. They always required much more
understanding of the entire compile/link/run cycle than any other
platform we use. Changing the CC line to cc_r works fine with the
only problem being the cascaded use of cc_r into any application
that links against libpq. Even with these complications, I think we
should still allow threading on AIX. Anyone who builds software on
AIX already knows how to manage these issues.

OK, but why does the AIX 5.1 version they have not have cc_r?

Because AIX 5.1 does not include cc_r; while it is termed the
"standard C compiler for AIX," there's about the same amount of
weasel-wording to that that there is to the notion of "open systems."

cc_r is a C compiler for AIX, and it evidently conforms (to some
degree or another) to the ANSI C standard.

It is not, however, included with AIX so as to allow it to be
considered a "standard component" that you can expect to find
available on AIX systems in general. It's only available if it has
been purchased.
--
"cbbrowne","@","ca.afilias.info"
<http://dev6.int.libertyrms.com/&gt;
Christopher Browne
(416) 673-4124 (land)

Zeugswetter Andreas DAZ SD wrote:

However, one thing we can do is to try this in Makefile.aix:
# AIX needs threads for everything that uses libpq
LIBS += $(PTHREAD_LIBS)
That is going to enable thread libs for all linking including the
backend, but it might work.

That is certainly wrong. The correct thing is to add PTHREAD_LIBS to
all and only those links that include libpq. I suspect that the cc_r

You mean like attached patch ? That is safe and can be applied.
My machine has problems with two configure tests with --enable-thread-safety.

1. the snprintf long long int check that uses int64 as variable name which is
a typedef in sys/inttypes.h

I changed it to ac_int64. Does that help?

2. whether strerror_r returns int check fails with a redefine of named function
from string.h

I see. Our test is:

int strerror_r(int, char *, size_t);

but AIX is:

int strerror_r(int, char *, int);

OK, I added an #ifndef _AIX to use 'int' for the 3rd arg for AIX.

No idea how to fix those, but workaround is easy :-(

-lpthread can be removed from PTHREAD_LIBS, the lib is a compat symlink to
-lpthreads, I guess it does not hurt eighter.

Yea, that is an effect of the tests we use and accumulate flags rather
than taking the first one that works. The test tests compiling but
creating libraries that use threads actually needs more than one flag on
some platforms so we just take all flags we can.

It would be nice if your fix also helped unixware but that needs
compilere flags for any thread library usage and we don't have
centralized libpq-specific compiler flags like we have library flags.

Please test this patch and report back. Thanks.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073