LD_LIBRARY_PATH versus rpath
Over at
http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php
we have a complaint about "make check" failing when the install is
intended to overwrite existing libraries (in particular, replacing
8.4 with 9.0 libpq). I've done some off-list investigation and
found that this appears to be a generic issue on Linux. pg_regress
invokes psql, which depends on libpq.so, and if psql fails due to
picking up the wrong libpq.so then you get behavior as described.
Now, pg_regress tries to ensure that the temporary installation
will work as desired by setting LD_LIBRARY_PATH to point at the
temp installation's lib/ directory. However, the psql executable
will by default get built with a DT_RPATH entry pointing at the
intended final installation lib/. And DT_RPATH overrides
LD_LIBRARY_PATH, in the Linux dynamic loader. man ld.so says:
The shared libraries needed by the program are searched for in the fol-
lowing order:
o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH attribute
does not exist. Use of DT_RPATH is deprecated.
o Using the environment variable LD_LIBRARY_PATH. Except if the exe-
cutable is a set-user-ID/set-group-ID binary, in which case it is
ignored.
o (ELF only) Using the directories specified in the DT_RUNPATH dynamic
section attribute of the binary if present.
o (etc etc)
Given that deprecation note, and the fact that what we're doing entirely
fails to work as desired, it seems like what we need to do is set
DT_RUNPATH instead of DT_RPATH. Further reading discloses that the
way to do that is to add "--enable-new-dtags" to the linker switches.
So the question is, should we modify Makefile.linux along the lines of
-rpath = -Wl,-rpath,'$(rpathdir)'
+rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtagsI asked around at Red Hat and was told that this would be unlikely
to have any negative side-effects, but I'm not sure how thoroughly
those guys thought about the consequences for non-mainstream Linux
machines. (In particular, I'm worried about really old distros
possibly not having this switch.)
My inclination is to try this in HEAD only and see if any problems
emerge during the beta cycle.
Comments?
regards, tom lane
On Thu, May 6, 2010 at 12:20 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Now, pg_regress tries to ensure that the temporary installation
will work as desired by setting LD_LIBRARY_PATH to point at the
temp installation's lib/ directory. However, the psql executable
will by default get built with a DT_RPATH entry pointing at the
intended final installation lib/. And DT_RPATH overrides
LD_LIBRARY_PATH, in the Linux dynamic loader. man ld.so says:
We only set RPATH if the install location isn't part of the default ld
library path specified by /etc/ld.so.conf right? Setting it if it is
in the default path would be antisocial.
--
greg
On ons, 2010-05-05 at 19:20 -0400, Tom Lane wrote:
Over at
http://archives.postgresql.org/pgsql-general/2010-05/msg00091.php
we have a complaint about "make check" failing when the install is
intended to overwrite existing libraries (in particular, replacing
8.4 with 9.0 libpq). I've done some off-list investigation and
found that this appears to be a generic issue on Linux. pg_regress
invokes psql, which depends on libpq.so, and if psql fails due to
picking up the wrong libpq.so then you get behavior as described.
Yeah, that's been broken since forever.
The shared libraries needed by the program are searched for in the fol-
lowing order:o (ELF only) Using the directories specified in the DT_RPATH dynamic
section attribute of the binary if present and DT_RUNPATH attribute
does not exist. Use of DT_RPATH is deprecated.o Using the environment variable LD_LIBRARY_PATH. Except if the exe-
cutable is a set-user-ID/set-group-ID binary, in which case it is
ignored.o (ELF only) Using the directories specified in the DT_RUNPATH dynamic
section attribute of the binary if present.
Ah, that sounds good.
So the question is, should we modify Makefile.linux along the lines of
-rpath = -Wl,-rpath,'$(rpathdir)' +rpath = -Wl,-rpath,'$(rpathdir)',--enable-new-dtags
I see this feature was added in 2001, so it should be OK to use.
My inclination is to try this in HEAD only and see if any problems
emerge during the beta cycle.
I wouldn't consider backpatching it at all.
On tor, 2010-05-06 at 11:20 +0100, Greg Stark wrote:
We only set RPATH if the install location isn't part of the default ld
library path specified by /etc/ld.so.conf right?
No. How would you determine that?
Setting it if it is
in the default path would be antisocial.
That's why there is --disable-rpath.
Greg Stark wrote
We only set RPATH if the install location isn't part of the default ld
library path specified by /etc/ld.so.conf right? Setting it if it is
in the default path would be antisocial.
How are we going to know at build time what is in the ld.soconf of the
installation machine?
cheers
andrew
Andrew Dunstan <andrew@dunslane.net> writes:
Greg Stark wrote
We only set RPATH if the install location isn't part of the default ld
library path specified by /etc/ld.so.conf right? Setting it if it is
in the default path would be antisocial.
How are we going to know at build time what is in the ld.soconf of the
installation machine?
Exactly. In practice, it's on the packager's head to specify
--disable-rpath if he intends to install into the platform's
regular library search path.
Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath
because "don't use RPATH" is part of the standard packaging guidelines
for that platform. However, pl/perl has to double back and use rpath
anyway because libperl.so doesn't exist in the ldconfig path; it's in
some version-numbered directory and they don't provide any link or
ldconfig entry so you could find it otherwise. Annoying as heck.
I've always wondered how many other packagers have to carry patches
similar to
http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch
regards, tom lane
On tor, 2010-05-06 at 09:38 -0400, Tom Lane wrote:
Funny point here: in the Fedora/RHEL RPMs, I use --disable-rpath
because "don't use RPATH" is part of the standard packaging guidelines
for that platform. However, pl/perl has to double back and use rpath
anyway because libperl.so doesn't exist in the ldconfig path; it's in
some version-numbered directory and they don't provide any link or
ldconfig entry so you could find it otherwise. Annoying as heck.
I've always wondered how many other packagers have to carry patches
similar to
http://cvs.fedoraproject.org/viewvc/rpms/postgresql/devel/postgresql-perl-rpath.patch
Debian has libperl in /usr/lib, so there is no issue. But if there
were, there is a relatively new policy that you can should use rpath if
you need a library that is installed in a nonstandard path. (Should
actually use this new runpath thing, perhaps.) The same new policy
prohibits packages from modifying /etc/ld.so.conf.