configure fails for perl check on CentOS8

On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:

Hello, While I'm moving to CentOS8 environment, I got stuck at
./configure with the following error.

configure: error: libperl library is requred for Perl

It complains that it needs -fPIC.

Configure uses only $Config{ccflags}, but it seems that
$Config{cccdlflags} is also required. The attached patch make
./configure success. (configure itself is excluded in the patch.)

./configure --with-perl

is working for me on Centos8 (double checked after a `dnf update`)

cheers

andrew

--

Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:

Hello, While I'm moving to CentOS8 environment, I got stuck at
./configure with the following error.
configure: error: libperl library is requred for Perl
It complains that it needs -fPIC.
Configure uses only $Config{ccflags}, but it seems that
$Config{cccdlflags} is also required. The attached patch make
./configure success. (configure itself is excluded in the patch.)

./configure --with-perl
is working for me on Centos8 (double checked after a `dnf update`)

Yeah, I'm quite suspicious of this too. Although we don't seem to have
any buildfarm members covering exactly RHEL8/CentOS8, we have enough
coverage of different Fedora releases to make it hard to believe that
we missed any changes in Red Hat's packaging of Perl.

Is this error perhaps occurring with a non-vendor Perl installation?
What's the exact error message? config.log might contain some useful
clues, too.

regards, tom lane

Hi. Sorry for the delay.

At Thu, 10 Oct 2019 11:51:21 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote in

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/10/19 1:46 AM, Kyotaro Horiguchi wrote:

Hello, While I'm moving to CentOS8 environment, I got stuck at
./configure with the following error.
configure: error: libperl library is requred for Perl
It complains that it needs -fPIC.
Configure uses only $Config{ccflags}, but it seems that
$Config{cccdlflags} is also required. The attached patch make
./configure success. (configure itself is excluded in the patch.)

./configure --with-perl
is working for me on Centos8 (double checked after a `dnf update`)

Yeah, I'm quite suspicious of this too. Although we don't seem to have
any buildfarm members covering exactly RHEL8/CentOS8, we have enough
coverage of different Fedora releases to make it hard to believe that
we missed any changes in Red Hat's packaging of Perl.

Is this error perhaps occurring with a non-vendor Perl installation?
What's the exact error message? config.log might contain some useful
clues, too.

The perl package is official one. I found the cause, that's my
mistake.

The problematic command line boils down to:

$ ./configure --with-perl CFLAGS=-O0

It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

By the way, is there any official way to specify options like -O0
while configure time?

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

Is this error perhaps occurring with a non-vendor Perl installation?
What's the exact error message? config.log might contain some useful
clues, too.

The perl package is official one. I found the cause, that's my
mistake.

The problematic command line boils down to:

$ ./configure --with-perl CFLAGS=-O0

It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

By the way, is there any official way to specify options like -O0
while configure time?

CFLAGS=-O0 configure --with-perl ...

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

The problematic command line boils down to:
$ ./configure --with-perl CFLAGS=-O0
It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

I don't quite understand why that wouldn't work.

By the way, is there any official way to specify options like -O0
while configure time?

CFLAGS=-O0 configure --with-perl ...

The way Horiguchi-san did it has been supported by autoconf for
a good long time now, so I don't think command line syntax is
the issue. Could that CFLAGS setting be interfering with some
feature test in our configure script?

regards, tom lane

On Wed, Oct 16, 2019 at 8:32 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On Tue, Oct 15, 2019 at 6:45 AM Kyotaro Horiguchi
<horikyota.ntt@gmail.com> wrote:

The problematic command line boils down to:
$ ./configure --with-perl CFLAGS=-O0
It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

I don't quite understand why that wouldn't work.

By the way, is there any official way to specify options like -O0
while configure time?

CFLAGS=-O0 configure --with-perl ...

The way Horiguchi-san did it has been supported by autoconf for
a good long time now, so I don't think command line syntax is
the issue.

Ah.

Could that CFLAGS setting be interfering with some
feature test in our configure script?

It looks like you need CFLAGS='-O0 -fPIC' on CentOS 8 when building with perl.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hello.

# I'm almost comming back..

At Wed, 16 Oct 2019 10:41:19 -0400, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote in

On Wed, Oct 16, 2019 at 8:32 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:

$ ./configure --with-perl CFLAGS=-O0
It is bash-aliased and survived without being found for a long time on
my Cent7 environment, but CentOS8 doesn't allow that.

I don't quite understand why that wouldn't work.

I'm confused with make CFLAGS=..

It looks like you need CFLAGS='-O0 -fPIC' on CentOS 8 when building with perl.

Yes, that is what my first mail said. I dug into it further.

The immediately problematic command generated by autoconf is:

gcc -o /tmp/conftest -Wall -Wmissing-prototypes -Wpointer-arith \
-Wdeclaration-after-statement -Werror=vla -Wendif-labels \
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing \
-fwrapv -fexcess-precision=standard -Wno-format-truncation \
-Wno-stringop-truncation \
-O0 \
-D_GNU_SOURCE -I/usr/lib64/perl5/CORE \
/tmp/conftest.c \
-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld \
-fstack-protector-strong -L/usr/local/lib -L/usr/lib64/perl5/CORE \
-lperl -lpthread -lresolv -ldl -lm -lcrypt -lutil -lc

/usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Very interestingly I don't get the error when the "-O0" is "-O2". It
is because gcc eliminates the PL_memory_wrap maybe by inlining.

The *final* problematic command boils down to:

gcc -o /tmp/conftest /tmp/conftest_O0.o \
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -lperl

That is, the problem doesn't happen without
"-specs=/usr/lib/rpm/redhat/redhat-hardened-ld".

I found the following bug report but it hasn't ever been fixed since
2016 on Fedora 24. I'm not sure about the newer versions.

https://bugzilla.redhat.com/show_bug.cgi?id=1343892

--
Kyotaro Horiguchi
NTT Open Source Software Center

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

The immediately problematic command generated by autoconf is:
...
/usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status

Very interestingly I don't get the error when the "-O0" is "-O2". It
is because gcc eliminates the PL_memory_wrap maybe by inlining.

Yeah, probably so. But I don't like the idea of fixing a problem
triggered by user-supplied CFLAGS by injecting more cflags from
elsewhere. That seems likely to be counterproductive, or at
least it risks overriding what the user wanted.

Can we fix this by using something other than perl_alloc() as
the tested-for function? That is surely a pretty arbitrary
choice. Are there any standard Perl entry points that are just
plain functions with no weird macro expansions?

regards, tom lane

On 10/18/19 9:50 AM, Tom Lane wrote:

Kyotaro Horiguchi <horikyota.ntt@gmail.com> writes:

The immediately problematic command generated by autoconf is:
...
/usr/bin/ld: /tmp/ccGxodNv.o: relocation R_X86_64_32 against symbol `PL_memory_wrap' can not be used when making a PIE object; recompile with -fPIC
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Very interestingly I don't get the error when the "-O0" is "-O2". It
is because gcc eliminates the PL_memory_wrap maybe by inlining.

Yeah, probably so. But I don't like the idea of fixing a problem
triggered by user-supplied CFLAGS by injecting more cflags from
elsewhere. That seems likely to be counterproductive, or at
least it risks overriding what the user wanted.

Can we fix this by using something other than perl_alloc() as
the tested-for function? That is surely a pretty arbitrary
choice. Are there any standard Perl entry points that are just
plain functions with no weird macro expansions?

I had a look in perl's proto.h but didn't see any obvious candidate. I
tried a couple of others (e.g. Perl_get_context) and got the same result
reported above.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/18/19 9:50 AM, Tom Lane wrote:

Can we fix this by using something other than perl_alloc() as
the tested-for function? That is surely a pretty arbitrary
choice. Are there any standard Perl entry points that are just
plain functions with no weird macro expansions?

I had a look in perl's proto.h but didn't see any obvious candidate. I
tried a couple of others (e.g. Perl_get_context) and got the same result
reported above.

I poked into this on a Fedora 30 installation and determined that the
stray reference is coming from this bit in Perl's inline.h:

/* saves machine code for a common noreturn idiom typically used in Newx*() */
GCC_DIAG_IGNORE_DECL(-Wunused-function);
static void
S_croak_memory_wrap(void)
{
Perl_croak_nocontext("%s",PL_memory_wrap);
}
GCC_DIAG_RESTORE_DECL;

Apparently, gcc is smart enough to optimize this away as unused ...
at any optimization level higher than -O0.  I confirmed that it works
at -O0 too, if you change this function declaration to "static inline"
--- but evidently, not doing so was intentional, so we won't get much
cooperation if we propose changing it (back?) to a plain static inline.

So the failure occurs just from reading this header, independently of
which particular Perl function we try to call; I'd supposed that there
was some connection between perl_alloc and PL_memory_wrap, but there
isn't.

I still don't much like the originally proposed patch. IMO it makes too
many assumptions about what is in Perl's ccflags, and perhaps more to the
point, it is testing libperl's linkability using switches we will not use
when we actually build plperl. So I think there's a pretty high risk of
breaking other cases if we fix it that way.

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way. Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

A less invasive idea is to forcibly add -O1 to CFLAGS for this autoconf
test. We'd have to be careful about doing so for non-gcc compilers, as
they might not understand that switch syntax ... but probably we could
get away with changing CFLAGS only when using a gcc-alike. Still, that's
a hack, and it doesn't have much to recommend it other than being more
localized.

regards, tom lane

On 10/20/19 1:23 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/18/19 9:50 AM, Tom Lane wrote:

Can we fix this by using something other than perl_alloc() as
the tested-for function? That is surely a pretty arbitrary
choice. Are there any standard Perl entry points that are just
plain functions with no weird macro expansions?

I had a look in perl's proto.h but didn't see any obvious candidate. I
tried a couple of others (e.g. Perl_get_context) and got the same result
reported above.

I poked into this on a Fedora 30 installation and determined that the
stray reference is coming from this bit in Perl's inline.h:

/* saves machine code for a common noreturn idiom typically used in Newx*() */
GCC_DIAG_IGNORE_DECL(-Wunused-function);
static void
S_croak_memory_wrap(void)
{
Perl_croak_nocontext("%s",PL_memory_wrap);
}
GCC_DIAG_RESTORE_DECL;

Apparently, gcc is smart enough to optimize this away as unused ...
at any optimization level higher than -O0.  I confirmed that it works
at -O0 too, if you change this function declaration to "static inline"
--- but evidently, not doing so was intentional, so we won't get much
cooperation if we propose changing it (back?) to a plain static inline.

So the failure occurs just from reading this header, independently of
which particular Perl function we try to call; I'd supposed that there
was some connection between perl_alloc and PL_memory_wrap, but there
isn't.

Yeah, I came to the same conclusion.

I still don't much like the originally proposed patch. IMO it makes too
many assumptions about what is in Perl's ccflags, and perhaps more to the
point, it is testing libperl's linkability using switches we will not use
when we actually build plperl. So I think there's a pretty high risk of
breaking other cases if we fix it that way.

Agreed.

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way. Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

Sounds like a plan. I agree it's annoying to have to do something large
for something so trivial.

A less invasive idea is to forcibly add -O1 to CFLAGS for this autoconf
test. We'd have to be careful about doing so for non-gcc compilers, as
they might not understand that switch syntax ... but probably we could
get away with changing CFLAGS only when using a gcc-alike. Still, that's
a hack, and it doesn't have much to recommend it other than being more
localized.

right. I think your other plan is better.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/20/19 1:23 PM, Tom Lane wrote:

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way. Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

Sounds like a plan. I agree it's annoying to have to do something large
for something so trivial.

Turns out it's not really that bad. We just have to transfer the
responsibility for setting CFLAGS_SL from the platform Makefiles
to the platform template files. (As a bonus, it'd be possible to
allow users to override CFLAGS_SL during configure, as they can
do for CFLAGS. But I didn't mess with that here.)

I checked that this fixes the Fedora build problem, but I've not
really tested it on any other platform. Still, there's not that
much to go wrong, one would think.

regards, tom lane

On 10/20/19 7:36 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/20/19 1:23 PM, Tom Lane wrote:

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way. Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

Sounds like a plan. I agree it's annoying to have to do something large
for something so trivial.

Turns out it's not really that bad. We just have to transfer the
responsibility for setting CFLAGS_SL from the platform Makefiles
to the platform template files. (As a bonus, it'd be possible to
allow users to override CFLAGS_SL during configure, as they can
do for CFLAGS. But I didn't mess with that here.)

I checked that this fixes the Fedora build problem, but I've not
really tested it on any other platform. Still, there's not that
much to go wrong, one would think.

LGTM

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Hello.

At Mon, 21 Oct 2019 08:29:39 -0400, Andrew Dunstan <andrew.dunstan@2ndquadrant.com> wrote in

On 10/20/19 7:36 PM, Tom Lane wrote:

Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:

On 10/20/19 1:23 PM, Tom Lane wrote:

The right way to fix it, likely, is to add CFLAGS_SL while performing this
particular autoconf test, as that would replicate the switches used for
plperl (and it turns out that adding -fPIC does solve this problem).
But the configure script doesn't currently know about CFLAGS_SL, so we'd
have to do some refactoring to approach it that way. Moving that logic
from the platform-specific Makefiles into configure doesn't seem
unreasonable, but it would make the patch bigger.

Sounds like a plan. I agree it's annoying to have to do something large
for something so trivial.

Turns out it's not really that bad. We just have to transfer the
responsibility for setting CFLAGS_SL from the platform Makefiles
to the platform template files. (As a bonus, it'd be possible to
allow users to override CFLAGS_SL during configure, as they can
do for CFLAGS. But I didn't mess with that here.)

I checked that this fixes the Fedora build problem, but I've not
really tested it on any other platform. Still, there's not that
much to go wrong, one would think.

LGTM

However it's done, but it looks good to me and actually the problem is
gone. Thaks!

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center