Debian Sid broke Perl

Noah Misch <noah@leadboat.com> writes:

The Debian Sid buildfarm members have dozens of failures over the last day,
because the latest Perl packages caused "perl -V:useshrplib" to report false.

Has anyone filed a bug report?

regards, tom lane

On Sat, Jun 06, 2020 at 06:38:47PM -0400, Tom Lane wrote:

Noah Misch <noah@leadboat.com> writes:

The Debian Sid buildfarm members have dozens of failures over the last day,
because the latest Perl packages caused "perl -V:useshrplib" to report false.

Has anyone filed a bug report?

Not me.

Noah Misch <noah@leadboat.com> writes:

On Sat, Jun 06, 2020 at 06:38:47PM -0400, Tom Lane wrote:

Noah Misch <noah@leadboat.com> writes:

The Debian Sid buildfarm members have dozens of failures over the last day,
because the latest Perl packages caused "perl -V:useshrplib" to report false.

Has anyone filed a bug report?

Not me.

A bit of searching turned up this:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798626

showing that the change was intentional. Somebody should push back on
that, but not being a Debian person it probably shouldn't be me.

regards, tom lane

I wrote:

A bit of searching turned up this:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798626

BTW, as far as I can tell from the underlying discussion at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=962138
there was no actual change in the existence of the shared
library, but what is now happening is that we are getting
a result reflecting the fact that /usr/bin/perl itself is
statically linked.

I wonder whether we could just drop the configure-time
test for useshrplib. The worst-case scenario is that a user
grinds through a build and eventually gets an obscure link error,
but that's probably quite uncommon these days.

regards, tom lane

On Sat, Jun 06, 2020 at 07:11:51PM -0400, Tom Lane wrote:

I wrote:

A bit of searching turned up this:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798626

BTW, as far as I can tell from the underlying discussion at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=962138
there was no actual change in the existence of the shared
library, but what is now happening is that we are getting
a result reflecting the fact that /usr/bin/perl itself is
statically linked.

Interesting.

I wonder whether we could just drop the configure-time
test for useshrplib. The worst-case scenario is that a user
grinds through a build and eventually gets an obscure link error,
but that's probably quite uncommon these days.

Losing that would not hurt much. This solution relies on all other Perl
configure tests getting the same answer from /usr/bin/perl that they would get
from /usr/bin/perl*gnu. thorntail currently does behave that way:

$ diff -u <(perl -MConfig -e 'print Config::config_sh') <(perl5.30-sparc64-linux-gnu -MConfig -e 'print Config::config_sh')
--- /dev/fd/63  2020-06-07 02:50:49.368000000 +0300
+++ /dev/fd/62  2020-06-07 02:50:49.368000000 +0300
@@ -103,14 +103,15 @@
 config_arg38='-Doptimize=-O2'
 config_arg39='-dEs'
 config_arg4='-Dcc=sparc64-linux-gnu-gcc'
-config_arg40='-Uuseshrplib'
+config_arg40='-Duseshrplib'
+config_arg41='-Dlibperl=libperl.so.5.30.3'
 config_arg5='-Dcpp=sparc64-linux-gnu-cpp'
 config_arg6='-Dld=sparc64-linux-gnu-gcc'
-config_arg7='-Dccflags=-DDEBIAN -DAPPLLIB_EXP="/etc/perl:/usr/lib/sparc64-linux-gnu/perl-base" -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-NCIX23/perl-5.30.3=. -fstack-protector-strong -Wformat -Werror=format-security'
+config_arg7='-Dccflags=-DDEBIAN -DAPPLLIB_EXP="/etc/perl" -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-NCIX23/perl-5.30.3=. -fstack-protector-strong -Wformat -Werror=format-security'
 config_arg8='-Dldflags= -Wl,-z,relro'
 config_arg9='-Dlddlflags=-shared -Wl,-z,relro'
-config_argc='40'
-config_args='-Dmksymlinks -Dusethreads -Duselargefiles -Dcc=sparc64-linux-gnu-gcc -Dcpp=sparc64-linux-gnu-cpp -Dld=sparc64-linux-gnu-gcc -Dccflags=-DDEBIAN -DAPPLLIB_EXP="/etc/perl:/usr/lib/sparc64-linux-gnu/perl-base" -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-NCIX23/perl-5.30.3=. -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=sparc64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.30 -Darchlib=/usr/lib/sparc64-linux-gnu/perl/5.30 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/sparc64-linux-gnu/perl5/5.30 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.30.3 -Dsitearch=/usr/local/lib/sparc64-linux-gnu/perl/5.30.3 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Ui_xlocale -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Uuseshrplib'
+config_argc='41'
+config_args='-Dmksymlinks -Dusethreads -Duselargefiles -Dcc=sparc64-linux-gnu-gcc -Dcpp=sparc64-linux-gnu-cpp -Dld=sparc64-linux-gnu-gcc -Dccflags=-DDEBIAN -DAPPLLIB_EXP="/etc/perl" -Wdate-time -D_FORTIFY_SOURCE=2 -g -O2 -fdebug-prefix-map=/build/perl-NCIX23/perl-5.30.3=. -fstack-protector-strong -Wformat -Werror=format-security -Dldflags= -Wl,-z,relro -Dlddlflags=-shared -Wl,-z,relro -Dcccdlflags=-fPIC -Darchname=sparc64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.30 -Darchlib=/usr/lib/sparc64-linux-gnu/perl/5.30 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/sparc64-linux-gnu/perl5/5.30 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.30.3 -Dsitearch=/usr/local/lib/sparc64-linux-gnu/perl/5.30.3 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Duse64bitint -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -Ui_libutil -Ui_xlocale -Uversiononly -DDEBUGGING=-g -Doptimize=-O2 -dEs -Duseshrplib -Dlibperl=libperl.so.5.30.3'
 contains='grep'
 cp='cp'
 cpio=''
@@ -919,7 +920,7 @@
 lib_ext='.a'
 libc='libc-2.31.so'
 libdb_needs_pthread='N'
-libperl='libperl.a'
+libperl='libperl.so.5.30'
 libpth='/usr/local/lib /usr/include/sparc64-linux-gnu /usr/lib /lib/sparc64-linux-gnu /lib/../lib /usr/lib/sparc64-linux-gnu /usr/lib/../lib /lib'
 libs='-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt'
 libsdirs=' /usr/lib/sparc64-linux-gnu'
@@ -1203,7 +1204,7 @@
 usequadmath='undef'
 usereentrant='undef'
 userelocatableinc='undef'
-useshrplib='false'
+useshrplib='true'
 usesitecustomize='undef'
 usesocks='undef'
 usethreads='define'

Noah Misch <noah@leadboat.com> writes:

On Sat, Jun 06, 2020 at 07:11:51PM -0400, Tom Lane wrote:

I wonder whether we could just drop the configure-time
test for useshrplib.

Losing that would not hurt much. This solution relies on all other Perl
configure tests getting the same answer from /usr/bin/perl that they would get
from /usr/bin/perl*gnu.

Aye, there's the rub.

thorntail currently does behave that way:

Does not, you mean? This part looks pretty fatal to the idea:

@@ -919,7 +920,7 @@
lib_ext='.a'
libc='libc-2.31.so'
libdb_needs_pthread='N'
-libperl='libperl.a'
+libperl='libperl.so.5.30'
libpth='/usr/local/lib /usr/include/sparc64-linux-gnu /usr/lib /lib/sparc64-linux-gnu /lib/../lib /usr/lib/sparc64-linux-gnu /usr/lib/../lib /lib'
libs='-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt'
libsdirs=' /usr/lib/sparc64-linux-gnu'

We can't accept linking plperl to the static libperl.a --- even if it
manages to work, which it won't on some hardware, that would result in
libperl becoming embedded in plperl.so. That breaks every rule of good
distribution management.

I fear we shall have to push back against this as a breaking change.
We can't realistically be expected to look for some non-default version
of perl to get our build settings from.

However ... if I'm reading perl.m4 correctly, what actually matters
here is what we get from

$PERL -MExtUtils::Embed -e ldopts

Could you double-check what that produces in each case?

regards, tom lane

On Sat, Jun 06, 2020 at 08:38:13PM -0400, Tom Lane wrote:

Noah Misch <noah@leadboat.com> writes:

On Sat, Jun 06, 2020 at 07:11:51PM -0400, Tom Lane wrote:

I wonder whether we could just drop the configure-time
test for useshrplib.

Losing that would not hurt much. This solution relies on all other Perl
configure tests getting the same answer from /usr/bin/perl that they would get
from /usr/bin/perl*gnu.

Aye, there's the rub.

thorntail currently does behave that way:

Does not, you mean? This part looks pretty fatal to the idea:

I meant that PostgreSQL's ./configure must get the same answers, and it does
(should have posted this instead of what I did post):

checking for PERL... perlwrap
configure: using perl 5.30.3
checking for Perl archlibexp... /usr/lib/sparc64-linux-gnu/perl/5.30
checking for Perl privlibexp... /usr/share/perl/5.30
checking for Perl useshrplib... true
checking for CFLAGS recommended by Perl... -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
checking for CFLAGS to compile embedded Perl... -DDEBIAN
checking for flags to link embedded Perl... -fstack-protector-strong -L/usr/local/lib -L/usr/lib/sparc64-linux-gnu/perl/5.30/CORE -lperl -ldl -lm -lpthread -lc -lcrypt

checking for PERL... perl5.30-sparc64-linux-gnu
configure: using perl 5.30.3
checking for Perl archlibexp... /usr/lib/sparc64-linux-gnu/perl/5.30
checking for Perl privlibexp... /usr/share/perl/5.30
checking for Perl useshrplib... true
checking for CFLAGS recommended by Perl... -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
checking for CFLAGS to compile embedded Perl... -DDEBIAN
checking for flags to link embedded Perl... -fstack-protector-strong -L/usr/local/lib -L/usr/lib/sparc64-linux-gnu/perl/5.30/CORE -lperl -ldl -lm -lpthread -lc -lcrypt

"perlwrap" is a script that fakes useshrplib:
#! /bin/sh
if [ "$*" = '-MConfig -e print $Config{useshrplib}' ]
then echo -n true
else exec perl "$@"
fi

Noah Misch <noah@leadboat.com> writes:

I meant that PostgreSQL's ./configure must get the same answers, and it does
(should have posted this instead of what I did post):

Ah, that looks good. I suppose that we can generally expect that the
ldflags output will look like "-L/some/path -lperl ...", and whether
or not the libperl in that directory is .so or .a is not going to affect
things at this level. Furthermore, given that this output is specifically
defined to be flags to be used to *embed* libperl, it's the distro's own
fault if they end up with libperl statically linked into other packages;
they should not be putting a .a-style library there.

So I'm content to fix this by removing the check for useshrplib.
Any objections?

regards, tom lane

I wrote:

So I'm content to fix this by removing the check for useshrplib.

Having said that ... it does not appear to me that the Debian perl
maintainer foresaw all the consequences of this change, so I went
ahead and filed some push-back at

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798626

I'll wait to see the reply before changing anything.

regards, tom lane

I wrote:

Having said that ... it does not appear to me that the Debian perl
maintainer foresaw all the consequences of this change, so I went
ahead and filed some push-back at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798626
I'll wait to see the reply before changing anything.

The maintainer says he'll revert the change, so I suppose the
buildfarm will go back to normal without extra effort on our part.

regards, tom lane

On Sun, Jun 07, 2020 at 11:06:27AM -0400, Tom Lane wrote:

The maintainer says he'll revert the change, so I suppose the
buildfarm will go back to normal without extra effort on our part.

The buildfarm has moved back to a green state as of the moment I am
writing this email (see serinus for example).
--
Michael