[PATCH 1/1] Fix detection of pwritev support for OSX.
Fixes:
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -c -o fd.o fd.c
fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]
part = pg_pwritev(fd, iov, iovcnt, offset);
^~~~~~~~~~
../../../../src/include/port/pg_iovec.h:49:20: note: expanded from macro 'pg_pwritev'
^~~~~~~
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include/sys/uio.h:104:9: note: 'pwritev' has been marked as being introduced in macOS 11.0
here, but the deployment target is macOS 10.15.0
ssize_t pwritev(int, const struct iovec *, int, off_t) __DARWIN_NOCANCEL(pwritev) __API_AVAILABLE(macos(11.0), ios(14.0), watchos(7.0), tvos(14.0));
^
fd.c:3661:10: note: enclose 'pwritev' in a __builtin_available check to silence this warning
part = pg_pwritev(fd, iov, iovcnt, offset);
^~~~~~~~~~
../../../../src/include/port/pg_iovec.h:49:20: note: expanded from macro 'pg_pwritev'
^~~~~~~
1 warning generated.
This results in a runtime error:
running bootstrap script ... dyld: lazy symbol binding failed: Symbol not found: _pwritev
Referenced from: /usr/local/pgsql/bin/postgres
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: _pwritev
Referenced from: /usr/local/pgsql/bin/postgres
Expected in: /usr/lib/libSystem.B.dylib
child process was terminated by signal 6: Abort trap: 6
To fix this we set -Werror=unguarded-availability-new so that a compile
test for pwritev will fail if the symbol is unavailable on the requested
SDK version.
---
configure | 88 ++++++++++++++++++++++++++++++++++++++++++++--------
configure.ac | 19 +++++++++++-
2 files changed, 93 insertions(+), 14 deletions(-)
diff --git a/configure b/configure
index 8af4b99021..503b0d27e6 100755
--- a/configure
+++ b/configure
@@ -5373,6 +5373,47 @@ if test x"$pgac_cv_prog_CC_cflags__Werror_vla" = x"yes"; then
fi+ # Prevent usage of symbols marked as newer than our target.
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports -Werror=unguarded-availability-new, for CFLAGS" >&5
+$as_echo_n "checking whether ${CC} supports -Werror=unguarded-availability-new, for CFLAGS... " >&6; }
+if ${pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ pgac_save_CFLAGS=$CFLAGS
+pgac_save_CC=$CC
+CC=${CC}
+CFLAGS="${CFLAGS} -Werror=unguarded-availability-new"
+ac_save_c_werror_flag=$ac_c_werror_flag
+ac_c_werror_flag=yes
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+int
+main ()
+{
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_compile "$LINENO"; then :
+ pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new=yes
+else
+ pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new=no
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+ac_c_werror_flag=$ac_save_c_werror_flag
+CFLAGS="$pgac_save_CFLAGS"
+CC="$pgac_save_CC"
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new" >&5
+$as_echo "$pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new" >&6; }
+if test x"$pgac_cv_prog_CC_cflags__Werror_unguarded_availability_new" = x"yes"; then
+ CFLAGS="${CFLAGS} -Werror=unguarded-availability-new"
+fi
+
+
# -Wvla is not applicable for C++ { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether ${CC} supports -Wendif-labels, for CFLAGS" >&5
@@ -15715,6 +15756,40 @@ $as_echo "#define HAVE_PS_STRINGS 1" >>confdefs.h
fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for pwritev" >&5
+$as_echo_n "checking for pwritev... " >&6; }
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#ifdef HAVE_SYS_UIO_H
+#include <sys/uio.h>
+#endif
+int
+main ()
+{
+struct iovec *iov;
+off_t offset;
+offset = 0;
+pwritev(0, iov, 0, offset);
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_compile "$LINENO"; then :
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+$as_echo "#define HAVE_PWRITEV 1" >>confdefs.h
+
+else
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+
ac_fn_c_check_func "$LINENO" "dlopen" "ac_cv_func_dlopen"
if test "x$ac_cv_func_dlopen" = xyes; then :
$as_echo "#define HAVE_DLOPEN 1" >>confdefs.h
@@ -15871,19 +15946,6 @@ esacfi
-ac_fn_c_check_func "$LINENO" "pwritev" "ac_cv_func_pwritev"
-if test "x$ac_cv_func_pwritev" = xyes; then :
- $as_echo "#define HAVE_PWRITEV 1" >>confdefs.h
-
-else
- case " $LIBOBJS " in
- *" pwritev.$ac_objext "* ) ;;
- *) LIBOBJS="$LIBOBJS pwritev.$ac_objext"
- ;;
-esac
-
-fi
-
ac_fn_c_check_func "$LINENO" "random" "ac_cv_func_random"
if test "x$ac_cv_func_random" = xyes; then :
$as_echo "#define HAVE_RANDOM 1" >>confdefs.h
diff --git a/configure.ac b/configure.ac
index 868a94c9ba..30fa39c859 100644
--- a/configure.ac
+++ b/configure.ac
@@ -494,6 +494,8 @@ if test "$GCC" = yes -a "$ICC" = no; then
AC_SUBST(PERMIT_DECLARATION_AFTER_STATEMENT)
# Really don't want VLAs to be used in our dialect of C
PGAC_PROG_CC_CFLAGS_OPT([-Werror=vla])
+ # Prevent usage of symbols marked as newer than our target.
+ PGAC_PROG_CC_CFLAGS_OPT([-Werror=unguarded-availability-new])
# -Wvla is not applicable for C++
PGAC_PROG_CC_CFLAGS_OPT([-Wendif-labels])
PGAC_PROG_CXX_CFLAGS_OPT([-Wendif-labels])
@@ -1726,6 +1728,22 @@ if test "$pgac_cv_var_PS_STRINGS" = yes ; then
AC_DEFINE([HAVE_PS_STRINGS], 1, [Define to 1 if the PS_STRINGS thing exists.])
fi+AC_MSG_CHECKING([for pwritev])
+AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
+[#ifdef HAVE_SYS_TYPES_H
+#include <sys/types.h>
+#endif
+#ifdef HAVE_SYS_UIO_H
+#include <sys/uio.h>
+#endif],
+[struct iovec *iov;
+off_t offset;
+offset = 0;
+pwritev(0, iov, 0, offset);
+])], [AC_MSG_RESULT(yes)
+AC_DEFINE([HAVE_PWRITEV], 1, [Define to 1 if you have the `pwritev' function.])],
+[AC_MSG_RESULT(no)])
+
AC_REPLACE_FUNCS(m4_normalize([
dlopen
explicit_bzero
@@ -1739,7 +1757,6 @@ AC_REPLACE_FUNCS(m4_normalize([
pread
preadv
pwrite
- pwritev
random
srandom
strlcat
--
2.30.0
James Hilliard <james.hilliard1@gmail.com> writes:
Fixes:
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -c -o fd.o fd.c
fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]
We already dealt with that by not selecting an SDK newer than the
underlying OS (see 4823621db). I do not believe that your proposal
is more reliable than that approach, and it's surely uglier. Are
we really going to abandon Autoconf's built-in checking method every
time Apple adds an API they should have had ten years ago? If so,
you forgot preadv ...
regards, tom lane
On Tue, Jan 19, 2021 at 8:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
Fixes:
gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unused-command-line-argument -O2 -I../../../../src/include -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -c -o fd.o fd.c
fd.c:3661:10: warning: 'pwritev' is only available on macOS 11.0 or newer [-Wunguarded-availability-new]We already dealt with that by not selecting an SDK newer than the
underlying OS (see 4823621db).
Tried that, doesn't work, not even sure how it could possibly fix this
issue at all,
this can not be fixed properly by selecting a specific SDK version
alone, it's the
symbols valid for a specific target deployment version that matters here.
I do not believe that your proposal
is more reliable than that approach, and it's surely uglier. Are
we really going to abandon Autoconf's built-in checking method every
time Apple adds an API they should have had ten years ago? If so,
you forgot preadv ...
I didn't run into an issue there for some reason...but this was the cleanest fix
I could come up with that seemed to work.
Show quoted text
regards, tom lane
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 8:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
We already dealt with that by not selecting an SDK newer than the
underlying OS (see 4823621db).
Tried that, doesn't work, not even sure how it could possibly fix this
issue at all,
It worked for me and for Sergey, so we need to figure out what's different
about your setup. What do you get from "xcrun --show-sdk-path" and
"xcrun --sdk macosx --show-sdk-path"? What have you got under
/Library/Developer/CommandLineTools/SDKs ?
this can not be fixed properly by selecting a specific SDK version
alone, it's the symbols valid for a specific target deployment version
that matters here.
I don't think I believe that argument. As a counterexample, supposing
that somebody were intentionally cross-compiling on an older OSX platform
but using a newer SDK, shouldn't they get an executable suited to the
SDK's target version?
(I realize that Apple thinks we ought to handle that through run-time
not compile-time adaptation, but I have no interest in going there.)
regards, tom lane
On Tue, Jan 19, 2021 at 8:57 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 8:27 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
We already dealt with that by not selecting an SDK newer than the
underlying OS (see 4823621db).Tried that, doesn't work, not even sure how it could possibly fix this
issue at all,It worked for me and for Sergey, so we need to figure out what's different
about your setup. What do you get from "xcrun --show-sdk-path" and
"xcrun --sdk macosx --show-sdk-path"? What have you got under
/Library/Developer/CommandLineTools/SDKs ?
$ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
$ ls -laht /Library/Developer/CommandLineTools/SDKs
total 0
drwxr-xr-x 5 root wheel 160B Jan 14 2020 .
drwxr-xr-x 8 root wheel 256B Jan 14 2020 MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224B Jan 14 2020 MacOSX10.14.sdk
lrwxr-xr-x 1 root wheel 15B Jan 14 2020 MacOSX.sdk -> MacOSX10.15.sdk
this can not be fixed properly by selecting a specific SDK version
alone, it's the symbols valid for a specific target deployment version
that matters here.I don't think I believe that argument. As a counterexample, supposing
that somebody were intentionally cross-compiling on an older OSX platform
but using a newer SDK, shouldn't they get an executable suited to the
SDK's target version?
Yep, that's exactly what this should fix:
MACOSX_DEPLOYMENT_TARGET=11.0 ./configure
checking for pwritev... yes
Which fails at runtime on 10.15:
dyld: lazy symbol binding failed: Symbol not found: _pwritev
Referenced from: /usr/local/pgsql/bin/postgres (which was built for
Mac OS X 11.0)
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: _pwritev
Referenced from: /usr/local/pgsql/bin/postgres (which was built for
Mac OS X 11.0)
Expected in: /usr/lib/libSystem.B.dylib
child process was terminated by signal 6: Abort trap: 6
MACOSX_DEPLOYMENT_TARGET=10.15 ./configure
checking for pwritev... no
Noticed a couple small issues, I'll send a v2.
Show quoted text
(I realize that Apple thinks we ought to handle that through run-time
not compile-time adaptation, but I have no interest in going there.)regards, tom lane
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 8:57 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
It worked for me and for Sergey, so we need to figure out what's different
about your setup. What do you get from "xcrun --show-sdk-path" and
"xcrun --sdk macosx --show-sdk-path"? What have you got under
/Library/Developer/CommandLineTools/SDKs ?
$ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
$ ls -laht /Library/Developer/CommandLineTools/SDKs
total 0
drwxr-xr-x 5 root wheel 160B Jan 14 2020 .
drwxr-xr-x 8 root wheel 256B Jan 14 2020 MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224B Jan 14 2020 MacOSX10.14.sdk
lrwxr-xr-x 1 root wheel 15B Jan 14 2020 MacOSX.sdk -> MacOSX10.15.sdk
Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that
is, it *does* give us a symlink to a 10.15 SDK) but by refusing to
believe we've got the right thing, we end up picking MacOSX11.1.sdk.
Drat. I suppose we could drop the heuristic about wanting a version
number in the SDK path, but I really don't want to do that. Now I'm
thinking about trying to dereference the symlink after the first step.
BTW, it's curious that you get a reference to the MacOSX.sdk symlink
where both Sergey and I got references to the actual directory.
Do you happen to recall the order in which you installed/upgraded
Xcode and its command line tools?
I don't think I believe that argument. As a counterexample, supposing
that somebody were intentionally cross-compiling on an older OSX platform
but using a newer SDK, shouldn't they get an executable suited to the
SDK's target version?
Yep, that's exactly what this should fix:
MACOSX_DEPLOYMENT_TARGET=11.0 ./configure
checking for pwritev... yes
Which fails at runtime on 10.15:
Well yeah, exactly. It should fail at run-time, because you
cross-compiled an executable that's not built for the machine
you're on. What we need is to prevent configure from setting up
a cross-compile situation by default.
regards, tom lane
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 8:57 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
It worked for me and for Sergey, so we need to figure out what's different
about your setup. What do you get from "xcrun --show-sdk-path" and
"xcrun --sdk macosx --show-sdk-path"? What have you got under
/Library/Developer/CommandLineTools/SDKs ?$ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
$ ls -laht /Library/Developer/CommandLineTools/SDKs
total 0
drwxr-xr-x 5 root wheel 160B Jan 14 2020 .
drwxr-xr-x 8 root wheel 256B Jan 14 2020 MacOSX10.15.sdk
drwxr-xr-x 7 root wheel 224B Jan 14 2020 MacOSX10.14.sdk
lrwxr-xr-x 1 root wheel 15B Jan 14 2020 MacOSX.sdk -> MacOSX10.15.sdkAh, got it. So "xcrun --show-sdk-path" tells us the right thing (that
is, it *does* give us a symlink to a 10.15 SDK) but by refusing to
believe we've got the right thing, we end up picking MacOSX11.1.sdk.
Drat. I suppose we could drop the heuristic about wanting a version
number in the SDK path, but I really don't want to do that. Now I'm
thinking about trying to dereference the symlink after the first step.
The MacOSX11.1.sdk can build for a 10.15 target just fine when passed
an appropriate MACOSX_DEPLOYMENT_TARGET, so that SDK should be
fine.
BTW, it's curious that you get a reference to the MacOSX.sdk symlink
where both Sergey and I got references to the actual directory.
Do you happen to recall the order in which you installed/upgraded
Xcode and its command line tools?
I generally just upgrade to the latest as it becomes available.
I don't think I believe that argument. As a counterexample, supposing
that somebody were intentionally cross-compiling on an older OSX platform
but using a newer SDK, shouldn't they get an executable suited to the
SDK's target version?Yep, that's exactly what this should fix:
MACOSX_DEPLOYMENT_TARGET=11.0 ./configure
checking for pwritev... yes
Which fails at runtime on 10.15:Well yeah, exactly. It should fail at run-time, because you
cross-compiled an executable that's not built for the machine
you're on. What we need is to prevent configure from setting up
a cross-compile situation by default.
The toolchain already selects the correct deployment target by default, the
issue is just that the configure test for pwritev was being done in a way that
ignored the deployment target version, I fixed that.
Show quoted text
regards, tom lane
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that
is, it *does* give us a symlink to a 10.15 SDK) but by refusing to
believe we've got the right thing, we end up picking MacOSX11.1.sdk.
Drat. I suppose we could drop the heuristic about wanting a version
number in the SDK path, but I really don't want to do that. Now I'm
thinking about trying to dereference the symlink after the first step.
The MacOSX11.1.sdk can build for a 10.15 target just fine when passed
an appropriate MACOSX_DEPLOYMENT_TARGET, so that SDK should be
fine.
But our out-of-the-box default should be to build for the current
platform; we don't want users to have to set MACOSX_DEPLOYMENT_TARGET
for that case. Besides, the problem we're having is exactly that Apple's
definition of "builds for a 10.15 target just fine" is different from
ours. They think you should use a run-time test not a compile-time test
to discover whether preadv is available, and we don't want to do that.
In almost all of the cases I've seen so far, Apple's compiler actually
does default to using an SDK matching the platform. The problem we
have is that we try to name the SDK explicitly, and the current
method is failing to pick the right one in your case. There are
several reasons for using an explicit -isysroot rather than just
letting the compiler default:
* We have seen cases in which the compiler acts as though it has
*no* default sysroot, and we have to help it out.
* The explicit root reduces version-skew build hazards for extensions
that are not built at the same time as the core system.
* There are a few tests in configure itself that need to know the
sysroot path to check for files there.
Anyway, the behavior you're seeing shows that 4823621db is still a
bit shy of a load. I'm thinking about the attached as a further
fix --- can you verify it helps for you?
regards, tom lane
Attachments:
further-fix-sysroot-heuristics.patchtext/x-diff; charset=us-ascii; name=further-fix-sysroot-heuristics.patchDownload+13-6
On Tue, Jan 19, 2021 at 1:54 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that
is, it *does* give us a symlink to a 10.15 SDK) but by refusing to
believe we've got the right thing, we end up picking MacOSX11.1.sdk.
Drat. I suppose we could drop the heuristic about wanting a version
number in the SDK path, but I really don't want to do that. Now I'm
thinking about trying to dereference the symlink after the first step.The MacOSX11.1.sdk can build for a 10.15 target just fine when passed
an appropriate MACOSX_DEPLOYMENT_TARGET, so that SDK should be
fine.But our out-of-the-box default should be to build for the current
platform; we don't want users to have to set MACOSX_DEPLOYMENT_TARGET
for that case. Besides, the problem we're having is exactly that Apple's
definition of "builds for a 10.15 target just fine" is different from
ours. They think you should use a run-time test not a compile-time test
to discover whether preadv is available, and we don't want to do that.
The default for MACOSX_DEPLOYMENT_TARGET is always the current
running OS version from my understanding. So if I build with MacOSX11.1.sdk
on 10.15 with default settings the binaries will work fine because the
MACOSX_DEPLOYMENT_TARGET gets set to 10.15 automatically even
if the same SDK is capable of producing incompatible binaries if you set
MACOSX_DEPLOYMENT_TARGET to 11.0.
In almost all of the cases I've seen so far, Apple's compiler actually
does default to using an SDK matching the platform. The problem we
have is that we try to name the SDK explicitly, and the current
method is failing to pick the right one in your case. There are
several reasons for using an explicit -isysroot rather than just
letting the compiler default:
No, it's only the MACOSX_DEPLOYMENT_TARGET that matches the
platform, SDK can be arbitrary more or less, but it will work fine because
the autoselected MACOSX_DEPLOYMENT_TARGET will force compatibility
no matter what SDK version you use. This is always how it has worked
from what I've seen.
* We have seen cases in which the compiler acts as though it has
*no* default sysroot, and we have to help it out.* The explicit root reduces version-skew build hazards for extensions
that are not built at the same time as the core system.
The deployment target is effectively entirely separate from SDK version,
so it really shouldn't make a difference unless the SDK is significantly
older or newer than the running version from what I can tell.
* There are a few tests in configure itself that need to know the
sysroot path to check for files there.Anyway, the behavior you're seeing shows that 4823621db is still a
bit shy of a load. I'm thinking about the attached as a further
fix --- can you verify it helps for you?
Best I can tell it provides no change for me(this patch is tested on top of it)
because it does not provide any MACOSX_DEPLOYMENT_TARGET
based feature detection for pwritev at all.
Show quoted text
regards, tom lane
On Tue, Jan 19, 2021 at 3:47 PM James Hilliard
<james.hilliard1@gmail.com> wrote:
On Tue, Jan 19, 2021 at 1:54 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 10:17 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ah, got it. So "xcrun --show-sdk-path" tells us the right thing (that
is, it *does* give us a symlink to a 10.15 SDK) but by refusing to
believe we've got the right thing, we end up picking MacOSX11.1.sdk.
Drat. I suppose we could drop the heuristic about wanting a version
number in the SDK path, but I really don't want to do that. Now I'm
thinking about trying to dereference the symlink after the first step.The MacOSX11.1.sdk can build for a 10.15 target just fine when passed
an appropriate MACOSX_DEPLOYMENT_TARGET, so that SDK should be
fine.But our out-of-the-box default should be to build for the current
platform; we don't want users to have to set MACOSX_DEPLOYMENT_TARGET
for that case. Besides, the problem we're having is exactly that Apple's
definition of "builds for a 10.15 target just fine" is different from
ours. They think you should use a run-time test not a compile-time test
to discover whether preadv is available, and we don't want to do that.The default for MACOSX_DEPLOYMENT_TARGET is always the current
running OS version from my understanding. So if I build with MacOSX11.1.sdk
on 10.15 with default settings the binaries will work fine because the
MACOSX_DEPLOYMENT_TARGET gets set to 10.15 automatically even
if the same SDK is capable of producing incompatible binaries if you set
MACOSX_DEPLOYMENT_TARGET to 11.0.In almost all of the cases I've seen so far, Apple's compiler actually
does default to using an SDK matching the platform. The problem we
have is that we try to name the SDK explicitly, and the current
method is failing to pick the right one in your case. There are
several reasons for using an explicit -isysroot rather than just
letting the compiler default:No, it's only the MACOSX_DEPLOYMENT_TARGET that matches the
platform, SDK can be arbitrary more or less, but it will work fine because
the autoselected MACOSX_DEPLOYMENT_TARGET will force compatibility
no matter what SDK version you use. This is always how it has worked
from what I've seen.* We have seen cases in which the compiler acts as though it has
*no* default sysroot, and we have to help it out.* The explicit root reduces version-skew build hazards for extensions
that are not built at the same time as the core system.The deployment target is effectively entirely separate from SDK version,
so it really shouldn't make a difference unless the SDK is significantly
older or newer than the running version from what I can tell.* There are a few tests in configure itself that need to know the
sysroot path to check for files there.Anyway, the behavior you're seeing shows that 4823621db is still a
bit shy of a load. I'm thinking about the attached as a further
fix --- can you verify it helps for you?Best I can tell it provides no change for me(this patch is tested on top of it)
because it does not provide any MACOSX_DEPLOYMENT_TARGET
based feature detection for pwritev at all.
Actually, this looks path looks wrong in general, the value for
"xcrun --sdk macosx --show-sdk-path" should take precedence over
"xcrun --show-sdk-path" as the latter may be used for IOS potentially.
On my system "xcodebuild -version -sdk macosx Path" and
"xcrun --sdk macosx --show-sdk-path" both point to the
correct latest MacOSX11.1.sdk SDK while "xcrun --show-sdk-path"
points to the older one.
Show quoted text
regards, tom lane
James Hilliard <james.hilliard1@gmail.com> writes:
Actually, this looks path looks wrong in general, the value for
"xcrun --sdk macosx --show-sdk-path" should take precedence over
"xcrun --show-sdk-path" as the latter may be used for IOS potentially.
What is "potentially"? I've found no direct means to control the
SDK path at all, but so far it appears that "xcrun --show-sdk-path"
agrees with the compiler's default -isysroot path as seen in the
compiler's -v output. I suspect that this isn't coincidental,
but reflects xcrun actually being used in the compiler launch
process. If it were to flip over to using a IOS SDK, that would
mean that bare "cc" would generate nonfunctional executables,
which just about any onlooker would agree is broken.
I'm really not excited about trying to make the build work with
a non-native SDK as you are proposing. I think that's just going
to lead to a continuing stream of problems, because of Apple's
opinions about how cross-version compatibility should work.
It also seems like unnecessary complexity, because there is always
(AFAICS) a native SDK version available. We just need to find it.
regards, tom lane
On Tue, Jan 19, 2021 at 6:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
Actually, this looks path looks wrong in general, the value for
"xcrun --sdk macosx --show-sdk-path" should take precedence over
"xcrun --show-sdk-path" as the latter may be used for IOS potentially.What is "potentially"?
Well I'm not sure the SDK parameter always defaults to macos although
I guess it probably does as I couldn't figure out a way to change it:
$ xcodebuild -showsdks
iOS SDKs:
iOS 14.3 -sdk iphoneos14.3
iOS Simulator SDKs:
Simulator - iOS 14.3 -sdk iphonesimulator14.3
macOS SDKs:
DriverKit 20.2 -sdk driverkit.macosx20.2
macOS 11.1 -sdk macosx11.1
tvOS SDKs:
tvOS 14.3 -sdk appletvos14.3
tvOS Simulator SDKs:
Simulator - tvOS 14.3 -sdk appletvsimulator14.3
watchOS SDKs:
watchOS 7.2 -sdk watchos7.2
watchOS Simulator SDKs:
Simulator - watchOS 7.2 -sdk watchsimulator7.2
I've found no direct means to control the
SDK path at all, but so far it appears that "xcrun --show-sdk-path"
agrees with the compiler's default -isysroot path as seen in the
compiler's -v output. I suspect that this isn't coincidental,
but reflects xcrun actually being used in the compiler launch
process. If it were to flip over to using a IOS SDK, that would
mean that bare "cc" would generate nonfunctional executables,
which just about any onlooker would agree is broken.
So there's some more weirdness involved here, whether or not you
have the command line install seems to affect the output of the
"xcrun --show-sdk-path" command, but not the
"xcrun --sdk macosx --show-sdk-path" command.
This is what I get without the command line tools:
$ xcrun --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
$ xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
this last one is just a symlink to the other path.
With command line tools this is different however:
$ xcrun --show-sdk-path
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
$ xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
Note that the /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
is different from the normal SDK and doesn't seem to be able to generate
binaries that target a 11.0 deployment target on my 10.15 system, however
I am unsure if this behavior can be relied upon.
So in terms of what works best, the newer normal SDK has the most flexibility
as it can produce both 10.15 target binaries and 11.0 target binaries
depending on the MACOSX_DEPLOYMENT_TARGET while the command
line tools SDK can only produce 10.15 target binaries it would appear.
Note that with my patch the binaries will always be compatible with the
host system by default, even if the SDK is capable of producing binaries
that are incompatible so building postgres works with and without the
command line tools SDK.
So I think "xcrun --sdk macosx --show-sdk-path" is probably preferable
but either should work as long as we can properly detect deployment
target symbol availability, regardless this SDK sysroot selection issue is
effectively an entirely different issue from the feature detection not properly
respecting the configured deployment target.
I'm really not excited about trying to make the build work with
a non-native SDK as you are proposing. I think that's just going
to lead to a continuing stream of problems, because of Apple's
opinions about how cross-version compatibility should work.
Well the minimum required target version is pretty much strictly based on
MACOSX_DEPLOYMENT_TARGET so our feature detection still needs
to use that, otherwise cross target compilation for newer or older targets will
not work correctly.
From my understanding the reason AC_REPLACE_FUNCS does not
throw an error for deployment target incompatible functions is that it only
checks if the function exists and not if it is actually useable, this is
why I had to add an explicit AC_LANG_PROGRAM compile test to
properly trigger a compile failure if the function is not usable for a
particular deployment target version, merely checking if the function
exists in the header is not sufficient.
It also seems like unnecessary complexity, because there is always
(AFAICS) a native SDK version available. We just need to find it.
Best I can tell this is not true, it is some(most?) of the time but
it's not something
we can rely upon as systems may only contain a newer SDK, but this newer SDK
is still capable of producing binaries that can run on the build host system so
this shouldn't be an issue as long as we can do target feature
detection properly.
Show quoted text
regards, tom lane
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 6:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I've found no direct means to control the
SDK path at all, but so far it appears that "xcrun --show-sdk-path"
agrees with the compiler's default -isysroot path as seen in the
compiler's -v output. I suspect that this isn't coincidental,
but reflects xcrun actually being used in the compiler launch
process. If it were to flip over to using a IOS SDK, that would
mean that bare "cc" would generate nonfunctional executables,
which just about any onlooker would agree is broken.
So there's some more weirdness involved here, whether or not you
have the command line install seems to affect the output of the
"xcrun --show-sdk-path" command, but not the
"xcrun --sdk macosx --show-sdk-path" command.
Yeah, that's what we discovered in the other thread. It seems that
with "--sdk macosx" you'll always get a pointer to the (solitary)
SDK under /Applications/Xcode.app, but with the short "xcrun
--show-sdk-path" command you might get either that or a pointer to
something under /Library/Developer/CommandLineTools.
I now believe what is actually happening with the short command is
that it's iterating through the available SDKs (according to some not
very clear search path) and picking the first one it finds that
matches the host system version. That matches the ktrace evidence
that shows it reading the SDKSettings.plist file in each SDK
directory. The fact that it can seize on either an actual directory
or an equivalent symlink might be due to chance ordering of directory
entries. (It'd be interesting to see "ls -f" output for your
/Library/Developer/CommandLineTools/SDKs directory ... though if
you've been experimenting with deinstall/reinstall, there's no
reason to suppose the entry order is still the same.)
I'm not sure that the case of not having the "command line tools"
installed is interesting for our purposes. AFAIK you have to have
that in order to have access to required tools like bison and gmake.
(That reminds me, I was intending to add something to our docs
about how-to-build-from-source to say that you need to install those.)
Note that with my patch the binaries will always be compatible with the
host system by default, even if the SDK is capable of producing binaries
that are incompatible so building postgres works with and without the
command line tools SDK.
Yeah. I don't see that as a benefit actually. Adding the
-no_weak_imports linker switch (or the other one you're suggesting)
means that you *cannot* cross-compile for a newer macOS version,
even if you set PG_SYSROOT and/or MACOSX_DEPLOYMENT_TARGET with the
intention of doing so. You'll still get a build that reflects the set
of kernel calls available on the host system. Admittedly, this is a
case that's not likely to be of interest to very many people, but
I don't see why a method with that restriction is superior to picking
a default SDK that matches the host system (and can be overridden).
So I think "xcrun --sdk macosx --show-sdk-path" is probably preferable
but either should work as long as we can properly detect deployment
target symbol availability, regardless this SDK sysroot selection issue is
effectively an entirely different issue from the feature detection not properly
respecting the configured deployment target.
No, I think it's pretty much equivalent. If we pick the right SDK
then we'll get the build we want.
regards, tom lane
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Tue, Jan 19, 2021 at 6:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I've found no direct means to control the
SDK path at all, but so far it appears that "xcrun --show-sdk-path"
agrees with the compiler's default -isysroot path as seen in the
compiler's -v output. I suspect that this isn't coincidental,
but reflects xcrun actually being used in the compiler launch
process. If it were to flip over to using a IOS SDK, that would
mean that bare "cc" would generate nonfunctional executables,
which just about any onlooker would agree is broken.So there's some more weirdness involved here, whether or not you
have the command line install seems to affect the output of the
"xcrun --show-sdk-path" command, but not the
"xcrun --sdk macosx --show-sdk-path" command.Yeah, that's what we discovered in the other thread. It seems that
with "--sdk macosx" you'll always get a pointer to the (solitary)
SDK under /Applications/Xcode.app, but with the short "xcrun
--show-sdk-path" command you might get either that or a pointer to
something under /Library/Developer/CommandLineTools.I now believe what is actually happening with the short command is
that it's iterating through the available SDKs (according to some not
very clear search path) and picking the first one it finds that
matches the host system version. That matches the ktrace evidence
that shows it reading the SDKSettings.plist file in each SDK
directory. The fact that it can seize on either an actual directory
or an equivalent symlink might be due to chance ordering of directory
entries. (It'd be interesting to see "ls -f" output for your
/Library/Developer/CommandLineTools/SDKs directory ... though if
Well at the moment I completely deleted that directory...and the build
works fine with my patch still.
you've been experimenting with deinstall/reinstall, there's no
reason to suppose the entry order is still the same.)I'm not sure that the case of not having the "command line tools"
installed is interesting for our purposes. AFAIK you have to have
that in order to have access to required tools like bison and gmake.
(That reminds me, I was intending to add something to our docs
about how-to-build-from-source to say that you need to install those.)
Yeah, not 100% sure but I was able to build just fine after deleting my
command line tools. I think it just switched to using the normal SDK
toolchain, I guess that's the fallback logic doing that.
It would be pretty annoying to have to install an outdated SDK just to
build postgres for no other reason than the autoconf feature detection
being broken.
Note that with my patch the binaries will always be compatible with the
host system by default, even if the SDK is capable of producing binaries
that are incompatible so building postgres works with and without the
command line tools SDK.Yeah. I don't see that as a benefit actually. Adding the
-no_weak_imports linker switch (or the other one you're suggesting)
means that you *cannot* cross-compile for a newer macOS version,
even if you set PG_SYSROOT and/or MACOSX_DEPLOYMENT_TARGET with the
intention of doing so.
Best I can tell this isn't true, I was able to cross compile for a newer
MACOSX_DEPLOYMENT_TARGET than my build host just fine. The
binary fails with a "Symbol not found: _pwritev" error when I try
to run it on the system that built it.
In regards to the -no_weak_imports switch...that is something different
from my understanding as it just strips the weak imports forcing the
fallback code paths to be taken instead, essentially functioning as if
the weak symbols are never available. It's largely separate from the
deployment target from my understanding as weak symbols are feature
that lets you use newer syscalls while still providing backwards
compatible fallbacks for older systems.
You'll still get a build that reflects the set
of kernel calls available on the host system. Admittedly, this is a
case that's not likely to be of interest to very many people, but
I don't see why a method with that restriction is superior to picking
a default SDK that matches the host system (and can be overridden).
But to fix the build when using a newer SDK overriding the SDK location
does not help, you would have to override the broken feature detection.
So I think "xcrun --sdk macosx --show-sdk-path" is probably preferable
but either should work as long as we can properly detect deployment
target symbol availability, regardless this SDK sysroot selection issue is
effectively an entirely different issue from the feature detection not properly
respecting the configured deployment target.No, I think it's pretty much equivalent. If we pick the right SDK
then we'll get the build we want.
Generally any recent SDK installed should work as long as the feature detection
in autoconf isn't broken. I'm not really sure what's the most correct option in
regards to picking a SDK version, however the feature detection should be
fixed regardless IMO.
Show quoted text
regards, tom lane
On 21.01.2021 02:07, Tom Lane wrote:
I now believe what is actually happening with the short command is
that it's iterating through the available SDKs (according to some not
very clear search path) and picking the first one it finds that
matches the host system version. That matches the ktrace evidence
that shows it reading the SDKSettings.plist file in each SDK
directory.
Yes, you are right. After some more digging...
It searches the DEVELOPER_DIR first and then
/Library/Developer/CommandLineTools, which is hardcoded.
My DEVELOPER_DIR is
% xcode-select -p
/Applications/Xcode.app/Contents/Developer
(For more detail try "otool -tV /usr/lib/libxcselect.dylib -p
_xcselect_get_developer_dir_path".)
It reads ProductVersion from
/System/Library/CoreServices/SystemVersion.plist
% plutil -p /System/Library/CoreServices/SystemVersion.plist | grep
ProductVersion
"ProductVersion" => "10.15.7"
Strips anything after the second dot, and prepends "macosx" to it, which
gives "macosx10.15".
Then it scans through SDK dirs looking up CanonicalName from
SDKSettings.plist until it finds a match with "macosx10.15".
The overall callstack:
% sudo dtrace -n 'syscall::getdirentries64:entry { ustack() }' -c 'xcrun
--show-sdk-path'
dtrace: description 'syscall::getdirentries64:entry ' matched 1 probe
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
dtrace: pid 20183 has exited
CPU ID FUNCTION:NAME
0 846 getdirentries64:entry
libsystem_kernel.dylib`__getdirentries64+0xa
libsystem_c.dylib`readdir$INODE64+0x23
libsystem_c.dylib`scandir$INODE64+0x6c
libxcrun.dylib`cltools_lookup_sdk_by_key+0x5f
libxcrun.dylib`cltools_lookup_boot_system_sdk+0xda
libxcrun.dylib`xcinfocache_resolve_sdkroot+0xc0
libxcrun.dylib`xcrun_main2+0x57a
libxcrun.dylib`xcrun_main+0x9
libxcselect.dylib`xcselect_invoke_xcrun_via_library+0xc8
libxcselect.dylib`xcselect_invoke_xcrun+0x25a
xcrun`DYLD-STUB$$getprogname
libdyld.dylib`start+0x1
xcrun`0x2
0 846 getdirentries64:entry
libsystem_kernel.dylib`__getdirentries64+0xa
libsystem_c.dylib`readdir$INODE64+0x23
libsystem_c.dylib`scandir$INODE64+0x6c
libxcrun.dylib`cltools_lookup_sdk_by_key+0x5f
libxcrun.dylib`cltools_lookup_boot_system_sdk+0xf3
libxcrun.dylib`xcinfocache_resolve_sdkroot+0xc0
libxcrun.dylib`xcrun_main2+0x57a
libxcrun.dylib`xcrun_main+0x9
libxcselect.dylib`xcselect_invoke_xcrun_via_library+0xc8
libxcselect.dylib`xcselect_invoke_xcrun+0x25a
xcrun`DYLD-STUB$$getprogname
libdyld.dylib`start+0x1
xcrun`0x2
The SDK search path:
% sudo dtrace -n 'pid$target:::entry
/probefunc=="cltools_lookup_sdk_by_key"/ { trace(copyinstr(arg0));
trace(copyinstr(arg1)) }' -c 'xcrun --show-sdk-path'
dtrace: description 'pid$target:::entry ' matched 17293 probes
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
dtrace: pid 20191 has exited
CPU ID FUNCTION:NAME
8 398290 cltools_lookup_sdk_by_key:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer
macosx10.15
9 398290 cltools_lookup_sdk_by_key:entry
/Library/Developer/CommandLineTools macosx10.15
The properties read from SDKSettings.plist:
% sudo dtrace -n 'pid$target:::entry
/probefunc=="_cltools_lookup_property_in_path"/ {
trace(copyinstr(arg0)); trace(copyinstr(arg1)); trace(copyinstr(arg2))
}' -c 'xcrun --show-sdk-path'
dtrace: description 'pid$target:::entry ' matched 17293 probes
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
dtrace: pid 20195 has exited
CPU ID FUNCTION:NAME
8 398288 _cltools_lookup_property_in_path:entry /
System/Library/CoreServices/SystemVersion.plist
ProductVersion
8 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/DriverKit20.2.sdk
SDKSettings.plist IsBaseSDK
8 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/DriverKit20.2.sdk
SDKSettings.plist CanonicalName
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/DriverKit20.2.sdk
SDKSettings.plist CanonicalNameForBuildSettings
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
SDKSettings.plist IsBaseSDK
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
SDKSettings.plist CanonicalName
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
SDKSettings.plist CanonicalNameForBuildSettings
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
SDKSettings.plist PLATFORM_NAME
4 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
SDKSettings.plist IsBaseSDK
2 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
SDKSettings.plist CanonicalName
2 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
SDKSettings.plist CanonicalNameForBuildSettings
2 398288 _cltools_lookup_property_in_path:entry
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk
SDKSettings.plist PLATFORM_NAME
2 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk SDKSettings.plist
IsBaseSDK
2 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk SDKSettings.plist
CanonicalName
2 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk SDKSettings.plist
CanonicalNameForBuildSettings
0 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk SDKSettings.plist
PLATFORM_NAME
0 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
SDKSettings.plist IsBaseSDK
0 398288 _cltools_lookup_property_in_path:entry
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk
SDKSettings.plist CanonicalName
BTW, on my machine /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
is skipped because it points to 11.0:
% ls -l /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
lrwxr-xr-x 1 root wheel 14 Nov 17 02:21
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -> MacOSX11.0.sdk
For more detail try
% otool -tV
/Applications/Xcode.app/Contents/Developer/usr/lib/libxcrun.dylib -p
_cltools_lookup_boot_system_sdk
James Hilliard <james.hilliard1@gmail.com> writes:
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm not sure that the case of not having the "command line tools"
installed is interesting for our purposes. AFAIK you have to have
that in order to have access to required tools like bison and gmake.
(That reminds me, I was intending to add something to our docs
about how-to-build-from-source to say that you need to install those.)
Yeah, not 100% sure but I was able to build just fine after deleting my
command line tools.
Hm. I've never been totally clear on what's included in the "command line
tools", although it's now apparent that one thing that gets installed is
an SDK matching the host OS version. However, Apple's description at [1]https://developer.apple.com/xcode/features/
says
Command Line Tools
Download the macOS SDK, headers, and build tools such as the Apple
LLVM compiler and Make. These tools make it easy to install open
source software or develop on UNIX within Terminal. macOS can
automatically download these tools the first time you try to build
software, and they are available on the downloads page.
which certainly strongly implies that gmake is not there otherwise.
At this point I lack any "bare" macOS system to check it on. I wonder
whether you have a copy of make available from MacPorts or Homebrew.
Or maybe uninstalling the command line tools doesn't really remove
everything?
It would be pretty annoying to have to install an outdated SDK just to
build postgres for no other reason than the autoconf feature detection
being broken.
It's only as "outdated" as your host system ;-). Besides, it doesn't
look like Apple's really giving you a choice not to.
The long and short of this is that I'm unwilling to buy into maintaining
our own substitutes for standard autoconf probes in order to make it
possible to use the wrong SDK version. The preadv/pwritev case is already
messy enough, and I fear that trying to support such scenarios is going to
lead to more and more pain in the future.
regards, tom lane
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm not sure that the case of not having the "command line tools"
installed is interesting for our purposes. AFAIK you have to have
that in order to have access to required tools like bison and gmake.
(That reminds me, I was intending to add something to our docs
about how-to-build-from-source to say that you need to install those.)Yeah, not 100% sure but I was able to build just fine after deleting my
command line tools.Hm. I've never been totally clear on what's included in the "command line
tools", although it's now apparent that one thing that gets installed is
an SDK matching the host OS version. However, Apple's description at [1]
saysCommand Line Tools
Download the macOS SDK, headers, and build tools such as the Apple
LLVM compiler and Make. These tools make it easy to install open
source software or develop on UNIX within Terminal. macOS can
automatically download these tools the first time you try to build
software, and they are available on the downloads page.which certainly strongly implies that gmake is not there otherwise.
At this point I lack any "bare" macOS system to check it on. I wonder
whether you have a copy of make available from MacPorts or Homebrew.
Or maybe uninstalling the command line tools doesn't really remove
everything?
Yeah, not entirely sure there but I do use homebrew.
It would be pretty annoying to have to install an outdated SDK just to
build postgres for no other reason than the autoconf feature detection
being broken.It's only as "outdated" as your host system ;-). Besides, it doesn't
look like Apple's really giving you a choice not to.
The newer SDK should work fine as long as long as the autoconf feature
detection is fixed somehow.
The long and short of this is that I'm unwilling to buy into maintaining
our own substitutes for standard autoconf probes in order to make it
possible to use the wrong SDK version. The preadv/pwritev case is already
messy enough, and I fear that trying to support such scenarios is going to
lead to more and more pain in the future.
Well it's actually a larger issue, if it isn't fixed then the ability
to change the
MACOSX_DEPLOYMENT_TARGET doesn't work properly, not only for
the case of having a newer SDK on an older host but it would also prevent
MACOSX_DEPLOYMENT_TARGET from working in general such as for
building with support for older targets from newer hosts, I'll see if there's
maybe a better way to fix the feature detection that's less of a maintenance
issue.
Show quoted text
regards, tom lane
On 22.01.2021 01:17, James Hilliard wrote:
On Thu, Jan 21, 2021 at 11:38 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
James Hilliard <james.hilliard1@gmail.com> writes:
On Wed, Jan 20, 2021 at 4:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm not sure that the case of not having the "command line tools"
installed is interesting for our purposes. AFAIK you have to have
that in order to have access to required tools like bison and gmake.
(That reminds me, I was intending to add something to our docs
about how-to-build-from-source to say that you need to install those.)Yeah, not 100% sure but I was able to build just fine after deleting my
command line tools.Hm. I've never been totally clear on what's included in the "command line
tools", although it's now apparent that one thing that gets installed is
an SDK matching the host OS version. However, Apple's description at [1]
saysCommand Line Tools
Download the macOS SDK, headers, and build tools such as the Apple
LLVM compiler and Make. These tools make it easy to install open
source software or develop on UNIX within Terminal. macOS can
automatically download these tools the first time you try to build
software, and they are available on the downloads page.which certainly strongly implies that gmake is not there otherwise.
At this point I lack any "bare" macOS system to check it on. I wonder
whether you have a copy of make available from MacPorts or Homebrew.
Or maybe uninstalling the command line tools doesn't really remove
everything?Yeah, not entirely sure there but I do use homebrew.
FWIW, I tested with a clean install of Catalina. Before I install
anything at all, I already have xcode-select, xcrun and all the shims in
/usr/bin for developer tools, including cc, make, git, xcodebuild...
Just about everything listed in the FILES section of "man xcode-select".
When I run any tool (except xcode-select), a GUI dialog pops up offering
to install the Command Line Tools. So apparently those shims are not
functional yet. I rejected the installation.
Instead I downloaded Xcode12.1.xip via [1]https://xcodereleases.com, the latest version with
macosx10.15 SDK. I unpacked it and installed by dragging Xcode.app to
/Applications. It seems to me there is no magic behind the scenes, just
moving the directory. I selectively checked that the shims in /usr/bin
didn't change after that.
Now, running "cc" tells me that I have to accept the Xcode license
agreement. After accepting it, all the shims in /usr/bin start to work,
forwarding to the real tools inside Xcode.app.
If I run the Homebrew installer, it says that it's going to install the
Command Line Tools. I don't know why it needs them, all the tools are
there already. I thought that CLT is a lighter-weight option when you
don't want the full Xcode installation, but Homebrew requires them anyway.
I rejected to install CLT and abandoned Homebrew. Then I just cloned and
built Postgres successfully. So it looks like Xcode is really enough, at
least on a recent macOS version.
[1]: https://xcodereleases.com
--
Sergey Shinderuk
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Sergey Shinderuk <s.shinderuk@postgrespro.ru> writes:
I rejected to install CLT and abandoned Homebrew. Then I just cloned and
built Postgres successfully. So it looks like Xcode is really enough, at
least on a recent macOS version.
Hm. I seem to recall having had to install CLT as well as Xcode back
in the day, but maybe Apple improved that. On the other side of the
coin, it also seems to be possible to build PG with only CLT and not
Xcode. I didn't try to verify that with a scorched-earth test, but
I did trash Xcode (and empty trash) on my wife's Mac, and I could
still build and "make check" with only the CLT in place.
[ pokes more carefully... ] Ah-hah, I see why I needed the CLT.
I bet you'll find that you can't build from "git clean -dfx" state
with only Xcode, because comparing the contents of
/Applications/Xcode.app/Contents/Developer/usr/bin and
/Library/Developer/CommandLineTools/usr/bin on my own Mac,
I observe that only the CLT provides bison and flex. I also see
install_name_tool only in the CLT; we don't depend on that today,
but may soon (see the latest thread about coping with SIP).
On the whole it looks like we should recommend installing the CLT
and not bothering with Xcode, which is about 10X the size:
$ du -hs /Library/Developer/CommandLineTools
1.1G /Library/Developer/CommandLineTools
$ du -hs /Applications/Xcode.app
15G /Applications/Xcode.app
regards, tom lane
Sergey Shinderuk <s.shinderuk@postgrespro.ru> writes:
If I run the Homebrew installer, it says that it's going to install the
Command Line Tools. I don't know why it needs them, all the tools are
there already. I thought that CLT is a lighter-weight option when you
don't want the full Xcode installation, but Homebrew requires them anyway.
BTW, reading [1]https://docs.brew.sh/Installation#3 I see
You can install Xcode, the CLT, or both; Homebrew supports all three
configurations.
So I'm not sure why you got that prompt, unless you were using a formula
that knew you were going to need bison.
regards, tom lane