Dynamic linking on AIX

This is a second try; this patch replaces
http://archives.postgresql.org/pgsql-patches/2006-09/msg00185.php

Incorporated are
- Tom Lane's suggestions for a more sane approach at
fixing Makefile.shlib
- Rocco Altier's additions to make the regression test run
by setting LIBPATH appropriately.
- Two changes in /src/interfaces/ecpg/test/Makefile.regress
and src/interfaces/ecpg/test/compat_informix/Makefile
without which 'make' fails if configure was called
with --disable-shared (this is not AIX-specific).

The line in src/makefiles/Makefile.aix
where I set 'libpath' also seems pretty ugly to me.
Do you have a better idea?

Basically I need to convert LDFLAGS like:
-L../../dir -L /extra/lib -lalib -Wl,yxz -L/more/libs
to
:/extra/lib:/more/libs
and couldn't think of a better way to do it.

It will fail if there is a -L path that contains
a blank :^(

Yours,
Laurenz Albe

I still would like to see a paragraph describing how AIX is different
from other platforms and what we are doing here.

---------------------------------------------------------------------------

Albe Laurenz wrote:

This is a second try; this patch replaces
http://archives.postgresql.org/pgsql-patches/2006-09/msg00185.php

Incorporated are
- Tom Lane's suggestions for a more sane approach at
fixing Makefile.shlib
- Rocco Altier's additions to make the regression test run
by setting LIBPATH appropriately.
- Two changes in /src/interfaces/ecpg/test/Makefile.regress
and src/interfaces/ecpg/test/compat_informix/Makefile
without which 'make' fails if configure was called
with --disable-shared (this is not AIX-specific).

The line in src/makefiles/Makefile.aix
where I set 'libpath' also seems pretty ugly to me.
Do you have a better idea?

Basically I need to convert LDFLAGS like:
-L../../dir -L /extra/lib -lalib -Wl,yxz -L/more/libs
to
:/extra/lib:/more/libs
and couldn't think of a better way to do it.

It will fail if there is a -L path that contains
a blank :^(

Yours,
Laurenz Albe

Content-Description: aixlink2.patch

[ Attachment, skipping... ]

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

I still would like to see a paragraph describing how AIX is different
from other platforms and what we are doing here.

Ok, I'll try to sum it up:

Shared libraries in AIX are different from shared libraries
in Linux.

A shared library on AIX is an 'ar' archive containing
shared objects.
A shared object is produced by the linker when invoked
appropriately (e.g. with -G), it is what we call a
shared library on Linux.

-> On AIX, you can do a static as well as a dynamic
-> link against a shared library, it depends on how you
-> invoke the linker.

When you link statically, the shared objects from
the library are added to your executable as required;
when you link dynamically, only references
to the shared objects are included in the executable.

Consequently you do not need a separate static library
on AIX if you have a dynamic library.

However, you CAN have static libraries (ar archives
containing *.o files), and the linker will link
against them. This will of course always be a
static link.

When the AIX linker searches for libraries to link,
it will look for a library libxy.a as well as for a
single shared object libxy.so when you tell it
to -lyx. When it finds both in the same directory,
it will prefer libpq.a unless invoked with -brtl.

This is where the problem occurs:

By default, PostgreSQL will (in the Linux way) create
a shared object libpq.so and a static library libpq.a
in the same directory.

Up to now, since the linker was invoked without the
-brtl flag, linking on AIX was always static, as the
linker preferred libpq.a over libpq.so.

We could have solved the problem by linking with
-brtl on AIX, but we chose to go a more AIX-conforming
way so that third party programs linking against
PostgreSQL libraries will not be fooled into
linking statically by default.

The 'new way' on AIX is:
- Create libxy.so.n as before from the static library
libxy.a with the linker.
- Remove libxy.a
- Recreate libxy.a as a dynamic library with
ar -cr libxy.a libxy.so.n
- Only install libxy.a, do not install libxy.so

Since linking is dynamic on AIX now, we have a new
problem:

We must make sure that the executable finds
its library even if the library is not installed in
one of the standard library paths (/usr/lib or /lib).

On Linux this is done with an RPATH, on AIX the
equivalent is LIBPATH that can be specified at link
time with -blibpath:<colon separated path> .
If you do not specify the LIBPATH, it is automatically
computed from the -L arguments given to the linker.
The LIBPATH, when set, must contain ALL directories where
shared libraries should be searched, including
the standard library directories.

Makefile.aix has been changed to link executables
with a LIBPATH that contains --libdir when PostgreSQL
is configured with --enable-rpath (the default).

The AIX equivalent for the Linux environment variable
LD_LIBRARY_PATH is LIBPATH.

The regression tests rely on LD_LIBRARY_PATH and have
to be changed to set LIBPATH as well.

I hope that's good enough,
Laurenz Albe

"Albe Laurenz" <all@adv.magwien.gv.at> writes:

This is a second try; this patch replaces
http://archives.postgresql.org/pgsql-patches/2006-09/msg00185.php

Looks good, applied.

The line in src/makefiles/Makefile.aix
where I set 'libpath' also seems pretty ugly to me.
It will fail if there is a -L path that contains
a blank :^(

We were already assuming no spaces in -L switches, see the $filter
manipulations in Makefile.shlib. So I simplified it to

libpath := $(shell echo $(subst -L,:,$(filter -L/%,$(LDFLAGS))) | sed -e's/ //g'):/usr/lib:/lib

It's annoying to have to shell out to sed to get rid of the spaces, but
this is gmake's fault for having such a brain-dead function call syntax.
After looking at the gmake manual, it is possible to use $subst to get
rid of spaces, but it's even uglier (and much harder to follow) than
the above ...

regards, tom lane

Great, I have added this to the bottom of the AIX FAQ. Thanks.

---------------------------------------------------------------------------

Albe Laurenz wrote:

I still would like to see a paragraph describing how AIX is different
from other platforms and what we are doing here.

Ok, I'll try to sum it up:

Shared libraries in AIX are different from shared libraries
in Linux.

A shared library on AIX is an 'ar' archive containing
shared objects.
A shared object is produced by the linker when invoked
appropriately (e.g. with -G), it is what we call a
shared library on Linux.

-> On AIX, you can do a static as well as a dynamic
-> link against a shared library, it depends on how you
-> invoke the linker.

When you link statically, the shared objects from
the library are added to your executable as required;
when you link dynamically, only references
to the shared objects are included in the executable.

Consequently you do not need a separate static library
on AIX if you have a dynamic library.

However, you CAN have static libraries (ar archives
containing *.o files), and the linker will link
against them. This will of course always be a
static link.

When the AIX linker searches for libraries to link,
it will look for a library libxy.a as well as for a
single shared object libxy.so when you tell it
to -lyx. When it finds both in the same directory,
it will prefer libpq.a unless invoked with -brtl.

This is where the problem occurs:

By default, PostgreSQL will (in the Linux way) create
a shared object libpq.so and a static library libpq.a
in the same directory.

Up to now, since the linker was invoked without the
-brtl flag, linking on AIX was always static, as the
linker preferred libpq.a over libpq.so.

We could have solved the problem by linking with
-brtl on AIX, but we chose to go a more AIX-conforming
way so that third party programs linking against
PostgreSQL libraries will not be fooled into
linking statically by default.

The 'new way' on AIX is:
- Create libxy.so.n as before from the static library
libxy.a with the linker.
- Remove libxy.a
- Recreate libxy.a as a dynamic library with
ar -cr libxy.a libxy.so.n
- Only install libxy.a, do not install libxy.so

Since linking is dynamic on AIX now, we have a new
problem:

We must make sure that the executable finds
its library even if the library is not installed in
one of the standard library paths (/usr/lib or /lib).

On Linux this is done with an RPATH, on AIX the
equivalent is LIBPATH that can be specified at link
time with -blibpath:<colon separated path> .
If you do not specify the LIBPATH, it is automatically
computed from the -L arguments given to the linker.
The LIBPATH, when set, must contain ALL directories where
shared libraries should be searched, including
the standard library directories.

Makefile.aix has been changed to link executables
with a LIBPATH that contains --libdir when PostgreSQL
is configured with --enable-rpath (the default).

The AIX equivalent for the Linux environment variable
LD_LIBRARY_PATH is LIBPATH.

The regression tests rely on LD_LIBRARY_PATH and have
to be changed to set LIBPATH as well.

I hope that's good enough,
Laurenz Albe

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

bruce@momjian.us (Bruce Momjian) writes:

Great, I have added this to the bottom of the AIX FAQ. Thanks.

It seems to me that this also warrants an entry in the release notes.

I'd think that an entry in the Source Code Changes section,
immediately after the "Reduce libraries linked into the backend
needlessly" might read:

* Add shared library support for AIX
--
let name="cbbrowne" and tld="linuxdatabases.info" in String.concat "@" [name;tld];;
http://linuxdatabases.info/info/wp.html
"I'd crawl over an acre of 'Visual This++' and 'Integrated Development
That' to get to gcc, Emacs, and gdb. Thank you."
-- Vance Petree, Virginia Power

Chris Browne wrote:

bruce@momjian.us (Bruce Momjian) writes:

Great, I have added this to the bottom of the AIX FAQ. Thanks.

It seems to me that this also warrants an entry in the release notes.

I'd think that an entry in the Source Code Changes section,
immediately after the "Reduce libraries linked into the backend
needlessly" might read:

* Add shared library support for AIX

Done.

--
Bruce Momjian bruce@momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +