Annoying build warnings from latest Apple toolchain

Started by Tom Laneover 2 years ago44 messageshackers
Jump to latest
#1Tom Lane
tgl@sss.pgh.pa.us

Since updating to Xcode 15.0, my macOS machines have been
spitting a bunch of linker-generated warnings. There
seems to be one instance of

ld: warning: -multiply_defined is obsolete

for each loadable module we link, and some program links complain

ld: warning: ignoring duplicate libraries: '-lpgcommon', '-lpgport'

You can see these in the build logs for both sifaka [1]https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=sifaka&dt=2023-09-26%2021%3A09%3A01&stg=build and
indri [2]https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=indri&dt=2023-09-26%2021%3A42%3A17&stg=build, so MacPorts isn't affecting it.

I'd held out some hope that this was a transient problem due to
the underlying OS still being Ventura, but I just updated another
machine to shiny new Sonoma (14.0), and it's still doing it.
Guess we gotta do something about it.

We used to need "-multiply_defined suppress" to suppress other
linker warnings. I tried removing that from the Makefile.shlib
recipes for Darwin, and those complaints go away while no new
ones appear, so that's good --- but I wonder whether slightly
older toolchain versions will still want it.

As for the duplicate libraries, yup guilty as charged, but
I think we were doing that intentionally to satisfy some other
old toolchains. I wonder whether removing the duplication
will create new problems.

regards, tom lane

[1]: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=sifaka&dt=2023-09-26%2021%3A09%3A01&stg=build
[2]: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=indri&dt=2023-09-26%2021%3A42%3A17&stg=build

#2Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#1)
Re: Annoying build warnings from latest Apple toolchain

I wrote:

Since updating to Xcode 15.0, my macOS machines have been
spitting a bunch of linker-generated warnings. There
seems to be one instance of
ld: warning: -multiply_defined is obsolete
for each loadable module we link ...

I poked into this a little more. We started using "-multiply_defined
suppress" in commit 9df308697 of 2004-07-13, which was in the OS X 10.3
era. I failed to find any specific discussion of that switch in our
archives, but the commit message suggests that I probably stole it
from a patch the Fink project was carrying.

Googling finds some non-authoritative claims that "-multiply_defined"
has been a no-op since OS X 10.9 (Mavericks). I don't have anything
older than 10.15 to check, but removing it on 10.15 does not seem
to cause any problems.

So I think we can safely just remove this switch from Makefile.shlib.
The meson build process isn't invoking it either I think.

The other thing will take a bit more work ...

regards, tom lane

#3Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#1)
Re: Annoying build warnings from latest Apple toolchain

I wrote:

Since updating to Xcode 15.0, my macOS machines have been
spitting a bunch of linker-generated warnings. ...
some program links complain

ld: warning: ignoring duplicate libraries: '-lpgcommon', '-lpgport'

I found that this is being caused by the libpq_pgport hack in
Makefile.global.in, which ensures that libpgcommon and libpgport
get linked before libpq. The comment freely admits that it results in
linking libpgcommon and libpgport twice. Now, AFAICS that whole hack
is unnecessary on any platform where we know how to do symbol export
control, because then libpq won't expose any of the troublesome
symbols to begin with. So we can resolve the problem by just not
doing that on macOS, as in the attached draft patch. I've confirmed
that this suppresses the duplicate-libraries warnings on Xcode 15.0
without creating any issues on older macOS (though I'm only in a
position to test as far back as Catalina).

The patch is written to change things only on macOS, but I wonder
if we should be more aggressive and change it for all platforms
where we have symbol export control (which is almost everything
these days). I doubt it'd make any noticeable difference in
build time, but doing that would give us more test coverage
which would help expose any weak spots.

I've not yet looked at the meson build infrastructure to
see if it needs a corresponding change.

regards, tom lane

Attachments:

v1-fix-duplicate-library-linkage.patchtext/x-diff; charset=us-ascii; name=v1-fix-duplicate-library-linkage.patchDownload+18-6
#4Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#3)
Re: Annoying build warnings from latest Apple toolchain

I wrote:

I've not yet looked at the meson build infrastructure to
see if it needs a corresponding change.

I think it doesn't, as long as all the relevant build targets
write their dependencies with "frontend_code" before "libpq".
(The need for this is, of course, documented nowhere. The state
of the documentation for our meson build system is abysmal.)

However, it's hard to test this, because the meson build
seems completely broken on current macOS:

-----
$ meson setup build --prefix=$HOME/pginstall
The Meson build system
Version: 0.64.1
Source dir: /Users/tgl/pgsql
Build dir: /Users/tgl/pgsql/build
Build type: native build
Project name: postgresql
Project version: 17devel

meson.build:9:0: ERROR: Unable to detect linker for compiler `cc -Wl,--version`
stdout:
stderr: ld: unknown options: --version
clang: error: linker command failed with exit code 1 (use -v to see invocation)

A full log can be found at /Users/tgl/pgsql/build/meson-logs/meson-log.txt
-----

That log file offers no more detail than the terminal output did.

(I also tried with a more recent meson version, 1.1.1, with
the same result.)

regards, tom lane

#5Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#4)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-27 16:52:44 -0400, Tom Lane wrote:

I wrote:

I've not yet looked at the meson build infrastructure to
see if it needs a corresponding change.

I think it doesn't, as long as all the relevant build targets
write their dependencies with "frontend_code" before "libpq".

Hm, that's not great. I don't think that should be required. I'll try to take
a look at why that's needed.

However, it's hard to test this, because the meson build
seems completely broken on current macOS:

I am travelling and I don't quite dare to upgrade my mac mini remotely. So I
can't try Sonoma directly.

But CI worked after switching to sonoma - although installing packages from
macports took forever, due to macports building all packages locally.

https://cirrus-ci.com/task/5133869171605504

There's some weird warnings about hashlib/blake2, but it looks like that's a
python installation issue. Looks like this is with python from macports in
PATH.

[00:59:14.442] ERROR:root:code for hash blake2b was not found.
[00:59:14.442] Traceback (most recent call last):
[00:59:14.442] File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/hashlib.py", line 307, in <module>
[00:59:14.442] globals()[__func_name] = __get_hash(__func_name)
[00:59:14.442] ^^^^^^^^^^^^^^^^^^^^^^^
[00:59:14.442] File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/hashlib.py", line 129, in __get_openssl_constructor
[00:59:14.442] return __get_builtin_constructor(name)
[00:59:14.442] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[00:59:14.442] File "/opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/hashlib.py", line 123, in __get_builtin_constructor
[00:59:14.442] raise ValueError('unsupported hash type ' + name)
[00:59:14.442] ValueError: unsupported hash type blake2b

This just happens whenever python's hashlib - supposedly in the standard
library - is imported.

There *are* some buildsystem warnings:
[00:59:27.289] [260/2328] Linking target src/interfaces/libpq/libpq.5.dylib
[00:59:27.289] ld: warning: -undefined error is deprecated
[00:59:27.289] ld: warning: ignoring -e, not used for output type

Full command:
[1/1] cc -o src/interfaces/libpq/libpq.5.dylib src/interfaces/libpq/libpq.5.dylib.p/fe-auth-scram.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-auth.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-connect.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-exec.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-lobj.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-misc.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-print.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-protocol3.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-secure.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-trace.c.o src/interfaces/libpq/libpq.5.dylib.p/legacy-pqsignal.c.o src/interfaces/libpq/libpq.5.dylib.p/libpq-events.c.o src/interfaces/libpq/libpq.5.dylib.p/pqexpbuffer.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-secure-common.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-secure-openssl.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-gssapi-common.c.o src/interfaces/libpq/libpq.5.dylib.p/fe-secure-gssapi.c.o -Wl,-dead_strip_dylibs -Wl,-headerpad_max_install_names -Wl,-undefined,error -shared -install_name @rpath/libpq.5.dylib -compatibility_version 5 -current_version 5.17 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk -Og -ggdb -Wl,-rpath,/opt/local/lib -Wl,-rpath,/opt/local/libexec/openssl3/lib src/common/libpgcommon_shlib.a src/port/libpgport_shlib.a -exported_symbols_list=/Users/admin/pgsql/build/src/interfaces/libpq/exports.list -lm /opt/local/lib/libintl.dylib /opt/local/lib/libgssapi_krb5.dylib /opt/local/lib/libldap.dylib /opt/local/lib/liblber.dylib /opt/local/libexec/openssl3/lib/libssl.dylib /opt/local/libexec/openssl3/lib/libcrypto.dylib /opt/local/lib/libz.dylib /opt/local/lib/libzstd.dylib
ld: warning: -undefined error is deprecated
ld: warning: ignoring -e, not used for output type

So we need to make the addition of -Wl,-undefined,error conditional, that
should be easy enough. Although I'm a bit confused about this being
deprecated.

For the -e bit, this seems to do the trick:
@@ -224,7 +224,7 @@ elif host_system == 'darwin'
library_path_var = 'DYLD_LIBRARY_PATH'

   export_file_format = 'darwin'
-  export_fmt = '-exported_symbols_list=@0@'
+  export_fmt = '-Wl,-exported_symbols_list,@0@'

mod_link_args_fmt = ['-bundle_loader', '@0@']
mod_link_with_dir = 'bindir'

It's quite annoying that apple is changing things option syntax.

(I also tried with a more recent meson version, 1.1.1, with
the same result.)

Looks like you need 1.2 for the new clang / ld output... Apparently apple's
linker changed the format of its version output :/.

Greetings,

Andres Freund

#6Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#5)
Re: Annoying build warnings from latest Apple toolchain

Andres Freund <andres@anarazel.de> writes:

On 2023-09-27 16:52:44 -0400, Tom Lane wrote:

I think it doesn't, as long as all the relevant build targets
write their dependencies with "frontend_code" before "libpq".

Hm, that's not great. I don't think that should be required. I'll try to take
a look at why that's needed.

Well, it's only important on platforms where we can't restrict
libpq.so from exporting all symbols. I don't know how close we are
to deciding that such cases are no longer interesting to worry about.
Makefile.shlib seems to know how to do it everywhere except Windows,
and I imagine we know how to do it over in the MSVC scripts.

However, it's hard to test this, because the meson build
seems completely broken on current macOS:

Looks like you need 1.2 for the new clang / ld output... Apparently apple's
linker changed the format of its version output :/.

Ah, yeah, updating MacPorts again brought in meson 1.2.1 which seems
to work. I now see a bunch of

ld: warning: ignoring -e, not used for output type
ld: warning: -undefined error is deprecated

which are unrelated. There's still one duplicate warning
from the backend link:

ld: warning: ignoring duplicate libraries: '-lpam'

I'm a bit baffled why that's showing up; there's no obvious
double reference to pam.

regards, tom lane

#7Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#6)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-28 16:46:08 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-27 16:52:44 -0400, Tom Lane wrote:

I think it doesn't, as long as all the relevant build targets
write their dependencies with "frontend_code" before "libpq".

Hm, that's not great. I don't think that should be required. I'll try to take
a look at why that's needed.

Well, it's only important on platforms where we can't restrict
libpq.so from exporting all symbols. I don't know how close we are
to deciding that such cases are no longer interesting to worry about.
Makefile.shlib seems to know how to do it everywhere except Windows,
and I imagine we know how to do it over in the MSVC scripts.

Hm, then I'd argue that we don't need to care about it anymore. The meson
build does the necessary magic on windows, as do the current msvc scripts.

I think right now it doesn't work as-is on sonoma, because apple decided to
change the option syntax, which is what causes the -e warning below, so the
relevant option is just ignored.

There's still one duplicate warning
from the backend link:

ld: warning: ignoring duplicate libraries: '-lpam'

I'm a bit baffled why that's showing up; there's no obvious
double reference to pam.

I think it's because multiple libraries/binaries depend on it. Meson knows how
to deduplicate libraries found via pkg-config (presumably because that has
enough information for a topological sort), but apparently not when they're
found as "raw" libraries. Until now that was also just pretty harmless, so I
understand not doing anything about it.

I see a way to avoid the warnings, but perhaps it's better to ask the meson
folks to put in a generic way of dealing with this.

Greetings,

Andres Freund

#8Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#7)
Re: Annoying build warnings from latest Apple toolchain

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 16:46:08 -0400, Tom Lane wrote:

Well, it's only important on platforms where we can't restrict
libpq.so from exporting all symbols. I don't know how close we are
to deciding that such cases are no longer interesting to worry about.
Makefile.shlib seems to know how to do it everywhere except Windows,
and I imagine we know how to do it over in the MSVC scripts.

Hm, then I'd argue that we don't need to care about it anymore. The meson
build does the necessary magic on windows, as do the current msvc scripts.

If we take that argument seriously, then I'm inclined to adjust my
upthread patch for Makefile.global.in so that it removes the extra
inclusions of libpgport/libpgcommon everywhere, not only macOS.
The rationale would be that it's not worth worrying about ABI
stability details on any straggler platforms.

I think right now it doesn't work as-is on sonoma, because apple decided to
change the option syntax, which is what causes the -e warning below, so the
relevant option is just ignored.

Hmm, we'd better fix that then. Or is it their bug? It looks to me like
clang's argument is -exported_symbols_list=/path/to/exports.list, so
it must be translating that to "-e".

regards, tom lane

#9Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#8)
Re: Annoying build warnings from latest Apple toolchain

I wrote:

Andres Freund <andres@anarazel.de> writes:

I think right now it doesn't work as-is on sonoma, because apple decided to
change the option syntax, which is what causes the -e warning below, so the
relevant option is just ignored.

Hmm, we'd better fix that then. Or is it their bug? It looks to me like
clang's argument is -exported_symbols_list=/path/to/exports.list, so
it must be translating that to "-e".

Looking closer, the documented syntax is

-exported_symbols_list filename

(two arguments, not one with an "="). That is what our Makefiles
use, and it still works fine with latest Xcode. However, meson.build
thinks it can get away with one argument containing "=", and evidently
that doesn't work now (or maybe it never did?).

I tried

export_fmt = '-exported_symbols_list @0@'

and

export_fmt = ['-exported_symbols_list', '@0@']

and neither of those did what I wanted, so maybe I will have to
study meson's command language sometime soon. In the meantime,
I suppose this might be an easy fix for somebody who knows their
way around meson.

regards, tom lane

#10Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#9)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-28 19:17:37 -0400, Tom Lane wrote:

I wrote:

Andres Freund <andres@anarazel.de> writes:

I think right now it doesn't work as-is on sonoma, because apple decided to
change the option syntax, which is what causes the -e warning below, so the
relevant option is just ignored.

Hmm, we'd better fix that then. Or is it their bug? It looks to me like
clang's argument is -exported_symbols_list=/path/to/exports.list, so
it must be translating that to "-e".

Looking closer, the documented syntax is

-exported_symbols_list filename

(two arguments, not one with an "="). That is what our Makefiles
use, and it still works fine with latest Xcode. However, meson.build
thinks it can get away with one argument containing "=", and evidently
that doesn't work now (or maybe it never did?).

It does still work on Ventura.

I tried

export_fmt = '-exported_symbols_list @0@'

That would expand to a single argument with a space inbetween.

and

export_fmt = ['-exported_symbols_list', '@0@']

That would work in many places, but not here, export_fmt is used as a format
string... We could make the callsites do that for each array element, but
there's an easier solution that seems to work for both Ventura and Sonoma -
however I don't have anything older to test with.

TBH, I find it hard to understand what arguments go to the linker and which to
the compiler on macos. The argument is documented for the linker and not the
compiler, but so far we'd been passing it to the compiler, so there must be
some logic forwarding it.

Looking through the clang code, I see various llvm libraries using
-Wl,-exported_symbols_list and there are tests
(clang/test/Driver/darwin-ld.c) ensuring both syntaxes work.

Thus the easiest fix looks to be to use this:

diff --git a/meson.build b/meson.build
index 5422885b0a2..16a2b0f801e 100644
--- a/meson.build
+++ b/meson.build
@@ -224,7 +224,7 @@ elif host_system == 'darwin'
   library_path_var = 'DYLD_LIBRARY_PATH'
   export_file_format = 'darwin'
-  export_fmt = '-exported_symbols_list=@0@'
+  export_fmt = '-Wl,-exported_symbols_list,@0@'

mod_link_args_fmt = ['-bundle_loader', '@0@']
mod_link_with_dir = 'bindir'

I don't have anything older than Ventura to check though.

Greetings,

Andres Freund

#11Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#10)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-28 19:20:27 -0700, Andres Freund wrote:

Thus the easiest fix looks to be to use this:

diff --git a/meson.build b/meson.build
index 5422885b0a2..16a2b0f801e 100644
--- a/meson.build
+++ b/meson.build
@@ -224,7 +224,7 @@ elif host_system == 'darwin'
library_path_var = 'DYLD_LIBRARY_PATH'
export_file_format = 'darwin'
-  export_fmt = '-exported_symbols_list=@0@'
+  export_fmt = '-Wl,-exported_symbols_list,@0@'

mod_link_args_fmt = ['-bundle_loader', '@0@']
mod_link_with_dir = 'bindir'

I don't have anything older than Ventura to check though.

Attached is the above change and a commit to change CI over to Sonoma. Not
sure when we should switch, but it seems useful to include for testing
purposes at the very least.

Greetings,

Andres Freund

Attachments:

v1-0001-meson-macos-Correct-exported_symbols_list-syntax-.patchtext/x-diff; charset=us-asciiDownload+1-3
v1-0002-ci-macos-Switch-to-Sonoma.patchtext/x-diff; charset=us-asciiDownload+3-4
#12Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#11)
Re: Annoying build warnings from latest Apple toolchain

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 19:20:27 -0700, Andres Freund wrote:

Thus the easiest fix looks to be to use this:
-  export_fmt = '-exported_symbols_list=@0@'
+  export_fmt = '-Wl,-exported_symbols_list,@0@'
I don't have anything older than Ventura to check though.

I don't have meson installed on my surviving Catalina box, but
I tried the equivalent thing in the Makefile universe:

diff --git a/src/Makefile.shlib b/src/Makefile.shlib
index f94d59d1c5..f2ed222cc7 100644
--- a/src/Makefile.shlib
+++ b/src/Makefile.shlib
@@ -136,7 +136,7 @@ ifeq ($(PORTNAME), darwin)
   BUILD.exports                = $(AWK) '/^[^\#]/ {printf "_%s\n",$$1}' $< >$@
   exports_file         = $(SHLIB_EXPORTS:%.txt=%.list)
   ifneq (,$(exports_file))
-    exported_symbols_list = -exported_symbols_list $(exports_file)
+    exported_symbols_list = -Wl,-exported_symbols_list,$(exports_file)
   endif
 endif

That builds and produces correctly-symbol-trimmed shlibs, so I'd
say it's fine. (Perhaps we should apply the above to HEAD
alongside the meson.build fix, to get more test coverage?)

Attached is the above change and a commit to change CI over to Sonoma. Not
sure when we should switch, but it seems useful to include for testing
purposes at the very least.

No opinion on when to switch CI. Sonoma is surely pretty bleeding edge
yet.

regards, tom lane

#13Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#12)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-28 22:53:09 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 19:20:27 -0700, Andres Freund wrote:

Thus the easiest fix looks to be to use this:
-  export_fmt = '-exported_symbols_list=@0@'
+  export_fmt = '-Wl,-exported_symbols_list,@0@'
I don't have anything older than Ventura to check though.

I don't have meson installed on my surviving Catalina box, but
I tried the equivalent thing in the Makefile universe:

diff --git a/src/Makefile.shlib b/src/Makefile.shlib
index f94d59d1c5..f2ed222cc7 100644
--- a/src/Makefile.shlib
+++ b/src/Makefile.shlib
@@ -136,7 +136,7 @@ ifeq ($(PORTNAME), darwin)
BUILD.exports                = $(AWK) '/^[^\#]/ {printf "_%s\n",$$1}' $< >$@
exports_file         = $(SHLIB_EXPORTS:%.txt=%.list)
ifneq (,$(exports_file))
-    exported_symbols_list = -exported_symbols_list $(exports_file)
+    exported_symbols_list = -Wl,-exported_symbols_list,$(exports_file)
endif
endif

That builds and produces correctly-symbol-trimmed shlibs, so I'd
say it's fine.

Thanks for testing!

I'll go and push that 16/HEAD then.

(Perhaps we should apply the above to HEAD alongside the meson.build fix, to
get more test coverage?)

The macos animals BF seem to run Ventura, so I think it'd not really provide
additional coverage that CI and your manual testing already has. So probably
not worth it from that angle?

Attached is the above change and a commit to change CI over to Sonoma. Not
sure when we should switch, but it seems useful to include for testing
purposes at the very least.

No opinion on when to switch CI. Sonoma is surely pretty bleeding edge
yet.

Yea, it does feel like that...

Greetings,

Andres Freund

#14Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#13)
Re: Annoying build warnings from latest Apple toolchain

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 22:53:09 -0400, Tom Lane wrote:

(Perhaps we should apply the above to HEAD alongside the meson.build fix, to
get more test coverage?)

The macos animals BF seem to run Ventura, so I think it'd not really provide
additional coverage that CI and your manual testing already has. So probably
not worth it from that angle?

My thought was that if it's in the tree we'd get testing from
non-buildfarm sources.

FWIW, I'm going to update sifaka/indri to Sonoma before too much longer
(they're already using Xcode 15.0 which is the Sonoma toolchain).
I recently pulled longfin up to Ventura, and plan to leave it on that
for the next year or so. I don't think anyone else is running macOS
animals.

regards, tom lane

#15Tristan Partin
tristan@neon.tech
In reply to: Andres Freund (#7)
Re: Annoying build warnings from latest Apple toolchain

On Thu Sep 28, 2023 at 5:22 PM CDT, Andres Freund wrote:

Hi,

On 2023-09-28 16:46:08 -0400, Tom Lane wrote:

There's still one duplicate warning
from the backend link:

ld: warning: ignoring duplicate libraries: '-lpam'

I'm a bit baffled why that's showing up; there's no obvious
double reference to pam.

I think it's because multiple libraries/binaries depend on it. Meson knows how
to deduplicate libraries found via pkg-config (presumably because that has
enough information for a topological sort), but apparently not when they're
found as "raw" libraries. Until now that was also just pretty harmless, so I
understand not doing anything about it.

I see a way to avoid the warnings, but perhaps it's better to ask the meson
folks to put in a generic way of dealing with this.

I wonder if this Meson PR[0]https://github.com/mesonbuild/meson/pull/12276 will help.

[0]: https://github.com/mesonbuild/meson/pull/12276

--
Tristan Partin
Neon (https://neon.tech)

#16Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#8)
Re: Annoying build warnings from latest Apple toolchain

I wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 16:46:08 -0400, Tom Lane wrote:

Well, it's only important on platforms where we can't restrict
libpq.so from exporting all symbols. I don't know how close we are
to deciding that such cases are no longer interesting to worry about.
Makefile.shlib seems to know how to do it everywhere except Windows,
and I imagine we know how to do it over in the MSVC scripts.

Hm, then I'd argue that we don't need to care about it anymore. The meson
build does the necessary magic on windows, as do the current msvc scripts.

If we take that argument seriously, then I'm inclined to adjust my
upthread patch for Makefile.global.in so that it removes the extra
inclusions of libpgport/libpgcommon everywhere, not only macOS.
The rationale would be that it's not worth worrying about ABI
stability details on any straggler platforms.

Looking closer, it's only since v16 that we have export list support
on all officially-supported platforms. Therefore, I think the prudent
thing to do in the back branches is use the patch I posted before,
to suppress the duplicate -l switches only on macOS. In HEAD,
I propose we simplify life by doing it everywhere, as attached.

regards, tom lane

Attachments:

v2-fix-duplicate-library-linkage.patchtext/x-diff; charset=us-ascii; name=v2-fix-duplicate-library-linkage.patchDownload+17-9
#17Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#16)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-29 12:14:40 -0400, Tom Lane wrote:

I wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 16:46:08 -0400, Tom Lane wrote:

Well, it's only important on platforms where we can't restrict
libpq.so from exporting all symbols. I don't know how close we are
to deciding that such cases are no longer interesting to worry about.
Makefile.shlib seems to know how to do it everywhere except Windows,
and I imagine we know how to do it over in the MSVC scripts.

Hm, then I'd argue that we don't need to care about it anymore. The meson
build does the necessary magic on windows, as do the current msvc scripts.

If we take that argument seriously, then I'm inclined to adjust my
upthread patch for Makefile.global.in so that it removes the extra
inclusions of libpgport/libpgcommon everywhere, not only macOS.
The rationale would be that it's not worth worrying about ABI
stability details on any straggler platforms.

Looking closer, it's only since v16 that we have export list support
on all officially-supported platforms.

Oh, right, before that Solaris didn't support it. I guess we could backpatch
that commit, it's simple enough, but I don't think I care enough about Solaris
to do so.

Therefore, I think the prudent thing to do in the back branches is use the
patch I posted before, to suppress the duplicate -l switches only on macOS.
In HEAD, I propose we simplify life by doing it everywhere, as attached.

Makes sense.

Greetings,

Andres Freund

#18Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#17)
Re: Annoying build warnings from latest Apple toolchain

Andres Freund <andres@anarazel.de> writes:

On 2023-09-29 12:14:40 -0400, Tom Lane wrote:

Looking closer, it's only since v16 that we have export list support
on all officially-supported platforms.

Oh, right, before that Solaris didn't support it. I guess we could backpatch
that commit, it's simple enough, but I don't think I care enough about Solaris
to do so.

HPUX would be an issue too, as we never did figure out how to do
export control on that. However, I doubt it would be a great idea
to back-patch export control in minor releases, even if we had
the patch at hand. That would be an ABI break of its own, although
it'd only affect clients that hadn't done things quite correctly.

Therefore, I think the prudent thing to do in the back branches is use the
patch I posted before, to suppress the duplicate -l switches only on macOS.
In HEAD, I propose we simplify life by doing it everywhere, as attached.

Makes sense.

Done that way. Were you going to push the -Wl,-exported_symbols_list
change?

regards, tom lane

#19Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#18)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-30 13:28:01 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-29 12:14:40 -0400, Tom Lane wrote:

Looking closer, it's only since v16 that we have export list support
on all officially-supported platforms.

Oh, right, before that Solaris didn't support it. I guess we could backpatch
that commit, it's simple enough, but I don't think I care enough about Solaris
to do so.

HPUX would be an issue too, as we never did figure out how to do
export control on that.

Ah, right.

However, I doubt it would be a great idea
to back-patch export control in minor releases, even if we had
the patch at hand. That would be an ABI break of its own, although
it'd only affect clients that hadn't done things quite correctly.

Agreed.

Therefore, I think the prudent thing to do in the back branches is use the
patch I posted before, to suppress the duplicate -l switches only on macOS.
In HEAD, I propose we simplify life by doing it everywhere, as attached.

Makes sense.

Done that way. Were you going to push the -Wl,-exported_symbols_list
change?

Done now.

Greetings,

Andres Freund

#20Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#14)
Re: Annoying build warnings from latest Apple toolchain

Hi,

On 2023-09-29 11:11:49 -0400, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2023-09-28 22:53:09 -0400, Tom Lane wrote:

(Perhaps we should apply the above to HEAD alongside the meson.build fix, to
get more test coverage?)

The macos animals BF seem to run Ventura, so I think it'd not really provide
additional coverage that CI and your manual testing already has. So probably
not worth it from that angle?

My thought was that if it's in the tree we'd get testing from
non-buildfarm sources.

I'm not against it, but it also doesn't quite seem necessary, given your
testing on Catalina.

FWIW, I'm going to update sifaka/indri to Sonoma before too much longer
(they're already using Xcode 15.0 which is the Sonoma toolchain).
I recently pulled longfin up to Ventura, and plan to leave it on that
for the next year or so. I don't think anyone else is running macOS
animals.

It indeed looks like nobody is. I wonder if it's worth setting up a gcc macos
animal, it's not too hard to imagine that breaking.

Greetings,

Andres Freund

#21Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#18)
#22Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#21)
#23Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#22)
#24Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#23)
#25Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#24)
#26Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#25)
#27Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#26)
#28Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#27)
#29Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#28)
#30Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#29)
#31Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#30)
#32Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#28)
#33Robert Haas
robertmhaas@gmail.com
In reply to: Tom Lane (#32)
#34Tom Lane
tgl@sss.pgh.pa.us
In reply to: Robert Haas (#33)
#35Tom Lane
tgl@sss.pgh.pa.us
In reply to: Tom Lane (#34)
#36Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#34)
#37Robert Haas
robertmhaas@gmail.com
In reply to: Andres Freund (#36)
#38Peter Eisentraut
peter_e@gmx.net
In reply to: Robert Haas (#37)
#39Aleksander Alekseev
aleksander@timescale.com
In reply to: Peter Eisentraut (#38)
#40Robert Haas
robertmhaas@gmail.com
In reply to: Peter Eisentraut (#38)
#41Andres Freund
andres@anarazel.de
In reply to: Robert Haas (#40)
#42Tom Lane
tgl@sss.pgh.pa.us
In reply to: Andres Freund (#41)
#43Andres Freund
andres@anarazel.de
In reply to: Tom Lane (#42)
#44Andres Freund
andres@anarazel.de
In reply to: Andres Freund (#43)