configure can't detect proper pthread flags

Max Filippov <jcmvbkbc@gmail.com> writes:

when PostgreSQL is cross-compiled in the Buildroot with uClibc toolchain
it may not correctly detect compiler/linker flags for threading. [1]
The reason is that config/acx_pthread.m4:146 uses compiler and linker
stdout and stderr to make decision if acx_pthread_ok should be yes or no:

if test "`(eval $ac_link 2>&1 1>&5)`" = "" && test "`(eval
$ac_compile 2>&1 1>&5)`" = ""; then

and the toolchain emits the following warning at linking step:

libcrypto.so: warning: gethostbyname is obsolescent, use
getnameinfo() instead.

git log doesn't tell much why it is done that way. Does anybody know?

The short answer is that the linker you're using is written by pedantic
idiots. Notice that the gethostbyname call it's complaining about is
somewhere inside libcrypto; it's *not* in Postgres, much less the test
program being built here. As such, we have no options whatever for
suppressing the complaint, which means that the complaint is being
delivered inappropriately, and it shouldn't be there. The linker is a
silly choice for a place to impose this sort of value judgment anyway;
it has absolutely no way to know whether the use of gethostbyname in
a particular program is unsafe.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

if test "`(eval $ac_link 2>&1 1>&5)`" = "" && test "`(eval
$ac_compile 2>&1 1>&5)`" = ""; then

FWIW, I happened to run into this recently on IRC with someone having
compile problems on FreeBSD (10.1); they were using some nonstandard
compile flags, and configure's pthread test was breaking as a result
(they did not report what the actual warning was).

While investigating that, I also noticed that this code prevents any
attempt at running configure with -x in effect from working properly,
making it a bit hard to debug.

--
Andrew (irc:RhodiumToad)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi Tom,

On Fri, Mar 20, 2015 at 3:50 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Max Filippov <jcmvbkbc@gmail.com> writes:

when PostgreSQL is cross-compiled in the Buildroot with uClibc toolchain
it may not correctly detect compiler/linker flags for threading. [1]
The reason is that config/acx_pthread.m4:146 uses compiler and linker
stdout and stderr to make decision if acx_pthread_ok should be yes or no:

if test "`(eval $ac_link 2>&1 1>&5)`" = "" && test "`(eval
$ac_compile 2>&1 1>&5)`" = ""; then

and the toolchain emits the following warning at linking step:

libcrypto.so: warning: gethostbyname is obsolescent, use
getnameinfo() instead.

git log doesn't tell much why it is done that way. Does anybody know?

The short answer is that the linker you're using is written by pedantic
idiots.

Well... That doesn't answer my question.

Notice that the gethostbyname call it's complaining about is
somewhere inside libcrypto; it's *not* in Postgres, much less the test
program being built here.

Actually it *is* in the program being built here, because it's being
linked with libcrypto. The full command line produced by the first eval
is this:

xtensa-linux-gcc -o conftest -Wall -Wmissing-prototypes
-Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
-fwrapv -fexcess-precision=standard -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mlongcalls
-mtext-section-literals -Os -pthread -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE conftest.c
-lssl -lcrypto -lz -lreadline -lrt -lcrypt -ldl -lm

and if I drop irrelevant libraries from that command its stdout+stderr
will probably be empty.

But I was curious why this test is written *that* way.

--
Thanks.
-- Max

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 04:51:55AM +0300, Max Filippov wrote:

xtensa-linux-gcc -o conftest -Wall -Wmissing-prototypes
-Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
-fwrapv -fexcess-precision=standard -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mlongcalls
-mtext-section-literals -Os -pthread -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE conftest.c
-lssl -lcrypto -lz -lreadline -lrt -lcrypt -ldl -lm

and if I drop irrelevant libraries from that command its stdout+stderr
will probably be empty.

But I was curious why this test is written *that* way.

Threading compiles are different for every platform so the code just
tries everything --- we didn't anticipate that adding a useless library
would actually cause a failure.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 5:09 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Fri, Mar 20, 2015 at 04:51:55AM +0300, Max Filippov wrote:

xtensa-linux-gcc -o conftest -Wall -Wmissing-prototypes
-Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
-fwrapv -fexcess-precision=standard -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mlongcalls
-mtext-section-literals -Os -pthread -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE conftest.c
-lssl -lcrypto -lz -lreadline -lrt -lcrypt -ldl -lm

and if I drop irrelevant libraries from that command its stdout+stderr
will probably be empty.

But I was curious why this test is written *that* way.

Threading compiles are different for every platform so the code just
tries everything --- we didn't anticipate that adding a useless library
would actually cause a failure.

Sorry, I must be not clear enough: why checking compiler/linker output
instead of checking their exit code or presence of produced object/
executable files?

--
Thanks.
-- Max

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

"Max" == Max Filippov <jcmvbkbc@gmail.com> writes:

Max> Sorry, I must be not clear enough: why checking compiler/linker
Max> output instead of checking their exit code or presence of produced
Max> object/ executable files?

Going by the comment some lines above, my guess would be "because some
compilers accept some option like -pthreads and issue a warning message
saying that it is ignored, and pg wants to not treat such options as
valid"

--
Andrew (irc:RhodiumToad)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 05:15:51AM +0300, Max Filippov wrote:

On Fri, Mar 20, 2015 at 5:09 AM, Bruce Momjian <bruce@momjian.us> wrote:

On Fri, Mar 20, 2015 at 04:51:55AM +0300, Max Filippov wrote:

xtensa-linux-gcc -o conftest -Wall -Wmissing-prototypes
-Wpointer-arith -Wdeclaration-after-statement -Wendif-labels
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing
-fwrapv -fexcess-precision=standard -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -mlongcalls
-mtext-section-literals -Os -pthread -D_LARGEFILE_SOURCE
-D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE conftest.c
-lssl -lcrypto -lz -lreadline -lrt -lcrypt -ldl -lm

and if I drop irrelevant libraries from that command its stdout+stderr
will probably be empty.

But I was curious why this test is written *that* way.

Threading compiles are different for every platform so the code just
tries everything --- we didn't anticipate that adding a useless library
would actually cause a failure.

Sorry, I must be not clear enough: why checking compiler/linker output
instead of checking their exit code or presence of produced object/
executable files?

Oh, uh, I don't rember the answer to that one. I think the code is in
config/acx_pthread.m4 and I assume we are just checking using the
configure macros that are provided.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andrew Gierth <andrew@tao11.riddles.org.uk> writes:

"Max" == Max Filippov <jcmvbkbc@gmail.com> writes:
Max> Sorry, I must be not clear enough: why checking compiler/linker
Max> output instead of checking their exit code or presence of produced
Max> object/ executable files?

Going by the comment some lines above, my guess would be "because some
compilers accept some option like -pthreads and issue a warning message
saying that it is ignored, and pg wants to not treat such options as
valid"

Precisely. We don't want every link step producing a useless warning.
Ideally, "make -s" would print nothing whatsoever; to the extent that
tools produce unsuppressable routine chatter, that's evil because it
makes it harder to notice actually-useful warnings.

(My current ambition in this area is to shut up clang from complaining
like so:
clang: warning: argument unused during compilation: '-pthread'
clang: warning: argument unused during compilation: '-pthread'
clang: warning: argument unused during compilation: '-pthread'
clang: warning: argument unused during compilation: '-pthread'
clang: warning: argument unused during compilation: '-pthread'
which is another bit of entirely useless pedantry, but rather hard to work
around because we assume that CFLAGS should be included when linking.)

It's tempting to consider avoiding Max's problem by doing the ACX_PTHREAD
test before picking up any other libraries. But I'm worried that that
would cause more problems than it solves. It's worth noting that the
Autoconf documentation specifically recommends testing for libraries
before testing for compiler characteristics.

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 5:20 AM, Andrew Gierth
<andrew@tao11.riddles.org.uk> wrote:

"Max" == Max Filippov <jcmvbkbc@gmail.com> writes:

Max> Sorry, I must be not clear enough: why checking compiler/linker
Max> output instead of checking their exit code or presence of produced
Max> object/ executable files?

Going by the comment some lines above, my guess would be "because some
compilers accept some option like -pthreads and issue a warning message
saying that it is ignored, and pg wants to not treat such options as
valid"

I've somehow missed that comment, thank you Andrew.

-- Max

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 6:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

We don't want every link step producing a useless warning.
Ideally, "make -s" would print nothing whatsoever; to the extent that
tools produce unsuppressable routine chatter, that's evil because it
makes it harder to notice actually-useful warnings.

Then maybe stderr tests should grep output for a specific option, the
one we're currently testing, not just any noise?

--
Thanks.
-- Max

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 7:01 AM, Max Filippov <jcmvbkbc@gmail.com> wrote:

On Fri, Mar 20, 2015 at 6:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

We don't want every link step producing a useless warning.
Ideally, "make -s" would print nothing whatsoever; to the extent that
tools produce unsuppressable routine chatter, that's evil because it
makes it harder to notice actually-useful warnings.

Then maybe stderr tests should grep output for a specific option, the
one we're currently testing, not just any noise?

That sounds awfully fragile to me. It can't really be safe to assume
we know precisely what the warning messages will look like. But it
seems to me that compiling every test program with every library we
might need is not a great plan.

(I don't know enough about autoconf to know whether changing that is realistic.)

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 08:05:48AM -0400, Robert Haas wrote:

On Fri, Mar 20, 2015 at 7:01 AM, Max Filippov <jcmvbkbc@gmail.com> wrote:

On Fri, Mar 20, 2015 at 6:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

We don't want every link step producing a useless warning.
Ideally, "make -s" would print nothing whatsoever; to the extent that
tools produce unsuppressable routine chatter, that's evil because it
makes it harder to notice actually-useful warnings.

Then maybe stderr tests should grep output for a specific option, the
one we're currently testing, not just any noise?

That sounds awfully fragile to me. It can't really be safe to assume
we know precisely what the warning messages will look like. But it
seems to me that compiling every test program with every library we
might need is not a great plan.

(I don't know enough about autoconf to know whether changing that is realistic.)

It was our only plan, and it has worked fine in the past. Someone is
going to have to do a lot of portability research to improve it.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On Fri, Mar 20, 2015 at 3:05 PM, Robert Haas <robertmhaas@gmail.com> wrote:

On Fri, Mar 20, 2015 at 7:01 AM, Max Filippov <jcmvbkbc@gmail.com> wrote:

On Fri, Mar 20, 2015 at 6:08 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

We don't want every link step producing a useless warning.
Ideally, "make -s" would print nothing whatsoever; to the extent that
tools produce unsuppressable routine chatter, that's evil because it
makes it harder to notice actually-useful warnings.

Then maybe stderr tests should grep output for a specific option, the
one we're currently testing, not just any noise?

That sounds awfully fragile to me. It can't really be safe to assume
we know precisely what the warning messages will look like.

Yes, I agree, not very good.

Ok, one more attempt: maybe instead of checking that stderr is empty
we could check that stderr has changed in the presence of the option
that we test?

--
Thanks.
-- Max

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Hi,

On 2015-03-20 03:14:48 +0300, Max Filippov wrote:

and the toolchain emits the following warning at linking step:

libcrypto.so: warning: gethostbyname is obsolescent, use
getnameinfo() instead.

FWIW, I think emitting such errors at link time is utterly pointless and
rather annoying. I can see a point of emitting them them when compiling
code that uses deprecated functions. But we quite obviously can't do
much about openssl using gethostbyname.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund wrote:

Hi,

On 2015-03-20 03:14:48 +0300, Max Filippov wrote:

and the toolchain emits the following warning at linking step:

libcrypto.so: warning: gethostbyname is obsolescent, use
getnameinfo() instead.

FWIW, I think emitting such errors at link time is utterly pointless and
rather annoying. I can see a point of emitting them them when compiling
code that uses deprecated functions. But we quite obviously can't do
much about openssl using gethostbyname.

We don't seem have much leverage with the guys producing the linker. If
we do, then surely our best bet is to get them to be quiet, or at least
provide an --yes-i-know-your-crap-is-noisy-please-shut-it-up option.

If we don't have leverage, and we really care enough about that platform
to want to work around this problem, it seems that the latest suggestion
of comparing the output of the linker with and without the option we're
testing (rather than just assuming that the output without the option
must surely be empty) is the safest bet ... It seems bad (fragile
hack), but let's see a patch and then we can judge.

Another option is to say "uclibc is broken beyond belief" and consider
it unsupported.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

On 2015-03-20 10:23:51 -0300, Alvaro Herrera wrote:

Andres Freund wrote:

FWIW, I think emitting such errors at link time is utterly pointless and
rather annoying. I can see a point of emitting them them when compiling
code that uses deprecated functions. But we quite obviously can't do
much about openssl using gethostbyname.

We don't seem have much leverage with the guys producing the linker. If
we do, then surely our best bet is to get them to be quiet, or at least
provide an --yes-i-know-your-crap-is-noisy-please-shut-it-up option.

It's not the linker, it's uclibc that adds the warning.

http://git.uclibc.org/uClibc/commit/?id=fdc6f045fa8b71a91a0c55b6390f8d0741e9f374

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Andres Freund wrote:

On 2015-03-20 10:23:51 -0300, Alvaro Herrera wrote:

Andres Freund wrote:

FWIW, I think emitting such errors at link time is utterly pointless and
rather annoying. I can see a point of emitting them them when compiling
code that uses deprecated functions. But we quite obviously can't do
much about openssl using gethostbyname.

We don't seem have much leverage with the guys producing the linker. If
we do, then surely our best bet is to get them to be quiet, or at least
provide an --yes-i-know-your-crap-is-noisy-please-shut-it-up option.

It's not the linker, it's uclibc that adds the warning.

http://git.uclibc.org/uClibc/commit/?id=fdc6f045fa8b71a91a0c55b6390f8d0741e9f374

Wow, that stuff has been there since 2009. So there's no way to shut it
up at all, is there.

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

By the way, acx-pthread.m4 has an outdated link to upstream
acx_pthread.m4. The correct link is

http://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=history;f=m4/ax_pthread.m4

--
�lvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

"Robert" == Robert Haas <robertmhaas@gmail.com> writes:

Then maybe stderr tests should grep output for a specific option,
the one we're currently testing, not just any noise?

Robert> That sounds awfully fragile to me. It can't really be safe to
Robert> assume we know precisely what the warning messages will look
Robert> like.

But how safe is it to assume that a warning message about option '-foo'
will contain the string '-foo' in it somewhere?

(though the trace output from -x still should be dealt with separately)

--
Andrew (irc:RhodiumToad)

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers