Re: databases/p5-postgresql-plperl links to wrong libperl.so

On Fri, Feb 11, 2005 at 04:41:55PM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 21:35 +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 11:10:15AM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 16:46 +0100, Palle Girgensohn wrote:

--On fredag, februari 11, 2005 10.24.22 -0500 Sven Willenberger
<sven@dmv.com> wrote:

FreeBSD 4.10
Postgresql 7.4.7
Perl 5.8.6_2 (from ports)

When building databases/p5-postgresql-plperl the resultant plperl.so
(/usr/local/lib/postgresql/plperl.so) links to the libperl.so
in /usr/lib instead of /usr/local/lib/perl5/5.8.6/mach/CORE/.

ldd /usr/local/lib/postgresql/plperl.so
/usr/local/lib/postgresql/plperl.so:
libperl.so => /usr/lib/libperl.so (0x2810b000)
libm.so.2 => /usr/lib/libm.so.2 (0x281a3000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x281be000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x281d7000)

2. _Or_ plperl does not go all the way to be a conformant perl-embedding
application. It looks at $Config{archlibexp}, but it does not follow
directions described in perlembed(1). In this case it's linking
should be fixed to respect that.

This does seem to be the case. I built postgresqql from source this time
rather than ports with ./configure --with-perl --with-openssl and, as
you point out, the congigure does find its way to the CORE directory but
the end product still links to the /usr/lib/libperl.so location.

Alright. It is not plperl folks fault, shared libraries and binaries do
things differently. Consider:

$ cat >binary.c
int main() {}
^D

$ gcc binary.c -Wl,-E -L/usr/local/lib \
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a \
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lcrypt -lutil \
-o binary

$ ldd ./binary
./binary:
libperl.so => /usr/lib/libperl.so (0x28066000)
libm.so.2 => /usr/lib/libm.so.2 (0x280fe000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x28119000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x28132000)
libc.so.4 => /usr/lib/libc.so.4 (0x2813b000)

$ cat >lib.c
int hello() { return 0; }
^D

$ gcc -fpic -DPIC -shared -Wl,-x,-soname,liblib.so.0 lib.c \
-Wl,-E -L/usr/local/lib \
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a \
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lcrypt -lutil \
-o liblib.so.0

$ ldd ./liblib.so.0
./liblib.so.0:
libperl.so => /usr/lib/libperl.so (0x28105000)
libm.so.2 => /usr/lib/libm.so.2 (0x2819d000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x281b8000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x281d1000)

Now, with -R things a-changing:

$ gcc -fpic -DPIC -shared -Wl,-x,-soname,liblib.so.0 lib.c \
-Wl,-E -L/usr/local/lib \
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a \
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lcrypt -lutil \
-R /usr/local/lib/perl5/5.6.2/mach/CORE \
-o liblib.so.0

$ ldd ./liblib.so.0
./liblib.so.0:
libperl.so => /usr/local/lib/perl5/5.6.2/mach/CORE/libperl.so (0x28105000)
libm.so.2 => /usr/lib/libm.so.2 (0x281ca000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x281e5000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x281fe000)

I am not that proficient with ld.so to know why it is like that, but the
easiest thing would be to persuade plperl to add that -R somewhere.

\Anton.
--
The moronity of the universe is a monotonically increasing function. --
Jarkko Hietaniemi

On Fri, Feb 11, 2005 at 11:09:35PM +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 04:41:55PM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 21:35 +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 11:10:15AM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 16:46 +0100, Palle Girgensohn wrote:

--On fredag, februari 11, 2005 10.24.22 -0500 Sven Willenberger
<sven@dmv.com> wrote:

FreeBSD 4.10
Postgresql 7.4.7
Perl 5.8.6_2 (from ports)

When building databases/p5-postgresql-plperl the resultant plperl.so
(/usr/local/lib/postgresql/plperl.so) links to the libperl.so
in /usr/lib instead of /usr/local/lib/perl5/5.8.6/mach/CORE/.

ldd /usr/local/lib/postgresql/plperl.so
/usr/local/lib/postgresql/plperl.so:
libperl.so => /usr/lib/libperl.so (0x2810b000)
libm.so.2 => /usr/lib/libm.so.2 (0x281a3000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x281be000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x281d7000)

2. _Or_ plperl does not go all the way to be a conformant perl-embedding
application. It looks at $Config{archlibexp}, but it does not follow
directions described in perlembed(1). In this case it's linking
should be fixed to respect that.

This does seem to be the case. I built postgresqql from source this time
rather than ports with ./configure --with-perl --with-openssl and, as
you point out, the congigure does find its way to the CORE directory but
the end product still links to the /usr/lib/libperl.so location.

Alright. It is not plperl folks fault, shared libraries and binaries do
things differently. Consider:

$ cat >binary.c
int main() {}
^D

$ gcc binary.c -Wl,-E -L/usr/local/lib \
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a \
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lcrypt -lutil \
-o binary

$ ldd ./binary
./binary:
libperl.so => /usr/lib/libperl.so (0x28066000)
libm.so.2 => /usr/lib/libm.so.2 (0x280fe000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x28119000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x28132000)
libc.so.4 => /usr/lib/libc.so.4 (0x2813b000)

Grrrr. What I wrote does not make any sense at all. :-(

It is plperl fault after all,

perl -MExtUtils::Embed -le ldopts

clearly states

-Wl,-R/usr/local/lib/perl5/5.6.2/mach/CORE -Wl,-E -L/usr/local/lib
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lc -lcrypt -lutil

Note the -R here, and the absense of it in the lines you posted.

Anton 'need more coffee or more sleep' Berezin
--
The moronity of the universe is a monotonically increasing function. --
Jarkko Hietaniemi

Sven Willenberger <sven@dmv.com> writes:

2. _Or_ plperl does not go all the way to be a conformant perl-embedding
application. It looks at $Config{archlibexp}, but it does not follow
directions described in perlembed(1). In this case it's linking
should be fixed to respect that.

You probably want to stick an rpath into the plperl.so library. PG 8.0
does that by default, but for 7.4 I recommend something like this:

RCS file: /projects/cvsroot/pgsql-server/src/pl/plperl/GNUmakefile,v
retrieving revision 1.10
diff -r1.10 GNUmakefile
18c18
< override CFLAGS := $(filter-out -Wall -Wmissing-declarations -Wmissing-prototypes, $(CFLAGS))
---

override CFLAGS := $(filter-out -Wall -Wmissing-declarations -Wmissing-prototypes, $(CFLAGS), -Wl,-rpath,$(perl_archlibexp)/CORE)

(The above is taken from the Red Hat RPMs and is known to work on Linux,
but I can't swear that it works on any other platform.)

regards, tom lane

On Fri, 2005-02-11 at 23:20 +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 11:09:35PM +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 04:41:55PM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 21:35 +0100, Anton Berezin wrote:

On Fri, Feb 11, 2005 at 11:10:15AM -0500, Sven Willenberger wrote:

On Fri, 2005-02-11 at 16:46 +0100, Palle Girgensohn wrote:

--On fredag, februari 11, 2005 10.24.22 -0500 Sven Willenberger
<sven@dmv.com> wrote:

FreeBSD 4.10
Postgresql 7.4.7
Perl 5.8.6_2 (from ports)

When building databases/p5-postgresql-plperl the resultant plperl.so
(/usr/local/lib/postgresql/plperl.so) links to the libperl.so
in /usr/lib instead of /usr/local/lib/perl5/5.8.6/mach/CORE/.

ldd /usr/local/lib/postgresql/plperl.so
/usr/local/lib/postgresql/plperl.so:
libperl.so => /usr/lib/libperl.so (0x2810b000)
libm.so.2 => /usr/lib/libm.so.2 (0x281a3000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x281be000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x281d7000)

2. _Or_ plperl does not go all the way to be a conformant perl-embedding
application. It looks at $Config{archlibexp}, but it does not follow
directions described in perlembed(1). In this case it's linking
should be fixed to respect that.

This does seem to be the case. I built postgresqql from source this time
rather than ports with ./configure --with-perl --with-openssl and, as
you point out, the congigure does find its way to the CORE directory but
the end product still links to the /usr/lib/libperl.so location.

Alright. It is not plperl folks fault, shared libraries and binaries do
things differently. Consider:

$ cat >binary.c
int main() {}
^D

$ gcc binary.c -Wl,-E -L/usr/local/lib \
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a \
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lcrypt -lutil \
-o binary

$ ldd ./binary
./binary:
libperl.so => /usr/lib/libperl.so (0x28066000)
libm.so.2 => /usr/lib/libm.so.2 (0x280fe000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x28119000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x28132000)
libc.so.4 => /usr/lib/libc.so.4 (0x2813b000)

Grrrr. What I wrote does not make any sense at all. :-(

It is plperl fault after all,

perl -MExtUtils::Embed -le ldopts

clearly states

-Wl,-R/usr/local/lib/perl5/5.6.2/mach/CORE -Wl,-E -L/usr/local/lib
/usr/local/lib/perl5/5.6.2/mach/auto/DynaLoader/DynaLoader.a
-L/usr/local/lib/perl5/5.6.2/mach/CORE -lperl -lm -lc -lcrypt -lutil

Note the -R here, and the absense of it in the lines you posted.

Anton 'need more coffee or more sleep' Berezin

Ok, I have found a couple lines in configure (4354-4356) and
config/perl.m4 (31-33) that confuse me a bit as it seems that their sole
purpose is to actually strip *out* the
-R/usr/local/lib/perl5/5.8.6/mach/CORE portion of the linker args:

pgac_tmp1=`$PERL -MExtUtils::Embed -e ldopts`
pgac_tmp2=`$PERL -MConfig -e 'print $Config{ccdlflags}'`
perl_embed_ldflags=`echo X"$pgac_tmp1" | sed "s/^X//;s%$pgac_tmp2%%"`

now pgac_tmp1 contains what should really be the ld args for the final
build/linking, pgac_tmp2 contains
-Wl,-R/usr/local/lib/perl5/5.8.6/mach/CORE

it looks like the sed line is stripping out pgac_tmp2 from the pgac_tmp1
leaving the system to use the default perl libperl.so.

when I modified perl_embed_ldflags to not remove the -R line, the
compile works and links to the proper libperl.so

Sven

P.S. From this point forward I will direct any followups to just the
postgresql mailing list as it does not appear to FreeBSD specific at
this point).

Sven Willenberger <sven@dmv.com> writes:

it looks like the sed line is stripping out pgac_tmp2 from the pgac_tmp1
leaving the system to use the default perl libperl.so.
when I modified perl_embed_ldflags to not remove the -R line, the
compile works and links to the proper libperl.so

It worked for you, but not necessarily for other people. This thread
may be instructive:
http://archives.postgresql.org/pgsql-bugs/2004-11/msg00181.php

regards, tom lane

--On fredag, februari 11, 2005 18.35.45 -0500 Tom Lane <tgl@sss.pgh.pa.us>
wrote:

Sven Willenberger <sven@dmv.com> writes:

it looks like the sed line is stripping out pgac_tmp2 from the pgac_tmp1
leaving the system to use the default perl libperl.so.
when I modified perl_embed_ldflags to not remove the -R line, the
compile works and links to the proper libperl.so

It worked for you, but not necessarily for other people. This thread
may be instructive:
http://archives.postgresql.org/pgsql-bugs/2004-11/msg00181.php

regards, tom lane

Instructive indeed, but it has no suggestion for how to fix this? It is not
quite true that this code has lived over two releas cycles without
complaints, because there have been som whining about it. In the case of
FreeBSD specifically, I doubt ppl will use anything but gcc, so we would
probably not need the sed line that removes ccdlflags, we could just opt it
out with a port's patch, I guess?

Something like
<http://archives.postgresql.org/pgsql-bugs/2004-11/msg00181.php&gt; would
probably work well for the FreeBSD port? Correct me I'm wrong.

/Palle

On Fri, 2005-02-11 at 18:35 -0500, Tom Lane wrote:

Sven Willenberger <sven@dmv.com> writes:

it looks like the sed line is stripping out pgac_tmp2 from the pgac_tmp1
leaving the system to use the default perl libperl.so.
when I modified perl_embed_ldflags to not remove the -R line, the
compile works and links to the proper libperl.so

It worked for you, but not necessarily for other people. This thread
may be instructive:
http://archives.postgresql.org/pgsql-bugs/2004-11/msg00181.php

regards, tom lane

I see they used the same solution that I did, namely remove the part of
the sed statement that removes the ccdflags frmo the ExtUtils::Embed's
output. The rpath solution you provided to patch GNUmakefile did not
work however, bailing with:

gcc -O2 -fno-strict-aliasing -Wmissing-declarations, -Wl,
-rpath, /usr/local/lib/perl5/5.8.6/mach/CORE -fpic -DPIC -I.
-I/usr/local/lib/perl5/5.8.6/mach/CORE -I../../../src/include -c -o
plperl.o plperl.c
gcc: cannot specify -o with -c or -S and multiple compilations

(As you said it may not work on all platforms).

At this point I will try to submit or suggest a patch for the freebsd
ports system to simply change the lines in the configure config/perl.m4
files to strip out the replacement part of the sed statement.

Thanks,

Sven

--On fredag, februari 11, 2005 19.08.54 -0500 Sven Willenberger
<sven@dmv.com> wrote:

On Fri, 2005-02-11 at 18:35 -0500, Tom Lane wrote:

Sven Willenberger <sven@dmv.com> writes:

it looks like the sed line is stripping out pgac_tmp2 from the
pgac_tmp1 leaving the system to use the default perl libperl.so.
when I modified perl_embed_ldflags to not remove the -R line, the
compile works and links to the proper libperl.so

It worked for you, but not necessarily for other people. This thread
may be instructive:
http://archives.postgresql.org/pgsql-bugs/2004-11/msg00181.php

regards, tom lane

I see they used the same solution that I did, namely remove the part of
the sed statement that removes the ccdflags frmo the ExtUtils::Embed's
output. The rpath solution you provided to patch GNUmakefile did not
work however, bailing with:

gcc -O2 -fno-strict-aliasing -Wmissing-declarations, -Wl,
-rpath, /usr/local/lib/perl5/5.8.6/mach/CORE -fpic -DPIC -I.
-I/usr/local/lib/perl5/5.8.6/mach/CORE -I../../../src/include -c -o
plperl.o plperl.c
gcc: cannot specify -o with -c or -S and multiple compilations

(As you said it may not work on all platforms).

At this point I will try to submit or suggest a patch for the freebsd
ports system to simply change the lines in the configure config/perl.m4
files to strip out the replacement part of the sed statement.

Thanks,

Sven

Sven, I think this sounds fine, beeing that it is FreeBSD specific. If you
can test this, once it's OK, send me a patch and I'll commit it.

Regards,
Palle

Sven Willenberger <sven@dmv.com> writes:

The rpath solution you provided to patch GNUmakefile did not
work however, bailing with:

gcc -O2 -fno-strict-aliasing -Wmissing-declarations, -Wl,
-rpath, /usr/local/lib/perl5/5.8.6/mach/CORE -fpic -DPIC -I.
-I/usr/local/lib/perl5/5.8.6/mach/CORE -I../../../src/include -c -o
plperl.o plperl.c
gcc: cannot specify -o with -c or -S and multiple compilations

If that's an accurate cut-and-paste, then you let a space sneak into
the -Wl,-rpath,path construct; perhaps that's the problem?

regards, tom lane

On Fri, 2005-02-11 at 19:54 -0500, Tom Lane wrote:

Sven Willenberger <sven@dmv.com> writes:

The rpath solution you provided to patch GNUmakefile did not
work however, bailing with:

gcc -O2 -fno-strict-aliasing -Wmissing-declarations, -Wl,
-rpath, /usr/local/lib/perl5/5.8.6/mach/CORE -fpic -DPIC -I.
-I/usr/local/lib/perl5/5.8.6/mach/CORE -I../../../src/include -c -o
plperl.o plperl.c
gcc: cannot specify -o with -c or -S and multiple compilations

If that's an accurate cut-and-paste, then you let a space sneak into
the -Wl,-rpath,path construct; perhaps that's the problem?

regards, tom lane

ahh, the linewrap issue of mailclients snuck in ... yes removing the
space did result in a successful build:

cc -O2 -pipe -march=pentiumpro -fno-strict-aliasing
-Wmissing-declarations, -Wl,-rpath,/usr/local/lib/perl5/5.8.6/mach/CORE
-fPIC -DPIC -I. -I/usr/local/lib/perl5/5.8.6/mach/CORE
-I../../../src/include -c -o plperl.o plperl.c
plperl.c: In function `compile_plperl_function':
plperl.c:541: warning: cast to pointer from integer of different size
plperl.c:730: warning: cast from pointer to integer of different size
cc: -rpath: linker input file unused since linking not done
cc: /usr/local/lib/perl5/5.8.6/mach/CORE: linker input file unused since
linking not done
cc -O2 -pipe -march=pentiumpro -fno-strict-aliasing
-Wmissing-declarations, -Wl,-rpath,/usr/local/lib/perl5/5.8.6/mach/CORE
-fPIC -DPIC -I. -I/usr/local/lib/perl5/5.8.6/mach/CORE
-I../../../src/include -c -o eloglvl.o eloglvl.c
cc: -rpath: linker input file unused since linking not done
cc: /usr/local/lib/perl5/5.8.6/mach/CORE: linker input file unused since
linking not done
/usr/bin/perl /usr/local/lib/perl5/5.8.6/ExtUtils/xsubpp
-typemap /usr/local/lib/perl5/5.8.6/ExtUtils/typemap SPI.xs >SPI.c
cc -O2 -pipe -march=pentiumpro -fno-strict-aliasing
-Wmissing-declarations, -Wl,-rpath,/usr/local/lib/perl5/5.8.6/mach/CORE
-fPIC -DPIC -I. -I/usr/local/lib/perl5/5.8.6/mach/CORE
-I../../../src/include -c -o SPI.o SPI.c
cc: -rpath: linker input file unused since linking not done
cc: /usr/local/lib/perl5/5.8.6/mach/CORE: linker input file unused since
linking not done
ar cr libplperl.a `lorder plperl.o eloglvl.o SPI.o | tsort`
ranlib libplperl.a
cc -O2 -pipe -march=pentiumpro -fno-strict-aliasing
-Wmissing-declarations, -Wl,-rpath,/usr/local/lib/perl5/5.8.6/mach/CORE
-fPIC -DPIC -shared -Wl,-x,-soname,libplperl.so.0 plperl.o eloglvl.o
SPI.o -L../../../src/port -L/usr/local/lib -Wl,-E
-L/usr/local/lib /usr/local/lib/perl5/5.8.6/mach/auto/DynaLoader/DynaLoader.a -L/usr/local/lib/perl5/5.8.6/mach/CORE -lperl -lm -lcrypt -lutil -R/usr/local/lib -o libplperl.so.0
rm -f libplperl.so
ln -s libplperl.so.0 libplperl.so

# cd work/postgresql-7.4.7/src/pl/plperl/
# ldd ./libplperl.so
./libplperl.so:
libperl.so => /usr/local/lib/perl5/5.8.6/mach/CORE/libperl.so
(0x2810b000)
libm.so.2 => /usr/lib/libm.so.2 (0x28207000)
libcrypt.so.2 => /usr/lib/libcrypt.so.2 (0x28222000)
libutil.so.3 => /usr/lib/libutil.so.3 (0x2823b000)

I have submitted a patch containing the other solution .. for what it's
worth attached is this solution as a patch. I will leave it to the
maintainer(s) to determine the "cleaner" solution :-)

Svem

Sven Willenberger <sven@dmv.com> writes:

plperl.c: In function `compile_plperl_function':
plperl.c:541: warning: cast to pointer from integer of different size
plperl.c:730: warning: cast from pointer to integer of different size

BTW, these scare the heck out of me. It looks to me like plperl.c is
casting a pointer to an integer in order to store it in a Perl hash.
Isn't there a better way to do that? Can anyone check on whether plperl
works at all on 64bit arches?

regards, tom lane

On Fri, 11 Feb 2005 20:40:18 -0500, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Sven Willenberger <sven@dmv.com> writes:

plperl.c: In function `compile_plperl_function':
plperl.c:541: warning: cast to pointer from integer of different size
plperl.c:730: warning: cast from pointer to integer of different size

BTW, these scare the heck out of me. It looks to me like plperl.c is
casting a pointer to an integer in order to store it in a Perl hash.
Isn't there a better way to do that? Can anyone check on whether plperl
works at all on 64bit arches?

It's been working fine for me on Gentoo Linux / AMD64.

--
Mike Rylander
mrylander@gmail.com
GPLS -- PINES Development
Database Developer
http://open-ils.org