Makefile.global will override configure parameters if "pgsql" and "postgres" appear anywhere in the source path name
Hi All,
Recently I was debugging an issue in which builds of postgresql
succeeded on a developer's workstation, but failed in our company's
build system.
After some investigation, I found that the value of pkglibdir (as used
by src/Makefile.global and various other places) was different even
though the ./configure invocations were identical. Eventually, I found
that Makefile.global was the culprit, with some sections like:
pkglibdir = $(libdir)
ifeq "$(findstring pgsql, $(pkglibdir))" ""
ifeq "$(findstring postgres, $(pkglibdir))" ""
override pkglibdir := $(pkglibdir)/postgresql
endif
endif
On the developer's workstation, the sources had been unpacked in a
directory, e.g. (the structure is a quirk convention of an internal
tool)
$HOME/src/postgresql13-13.7/source
On the production build system, the path used was:
/build/source
The --prefix used is a subdirectory of the source directory (in this
context we're only building the source to get the static libpq and
headers, which we use to build another piece of software).
On the developer's workstation, the string "postgres" is part of the
path, and thus pkglibdir does not get "postgresql" appended via an
override (nor any of the other variables). But on the build system,
both tests pass, and the override with appended "postgresql" do occur.
I'm assuming that the intention of findstring postgres was to tidy up
values like pkglibdir when libdir is e.g. /usr/lib/postgres/, which
makes sense, but not when "postgres" appears in libdir for some other
reason.
We're able to patch around this, but I thought it might be worth
reporting. I considered devising a patch but haven't yet thought of a
good strategy, and I suspect it might be trick to avoid breaking other
users who may (even unwittingly) rely on this quirk. I would be
delighted to accept any suggestions however.
Thanks!
.andy
On 2022-Jul-15, Andy Bailey wrote:
After some investigation, I found that the value of pkglibdir (as used
by src/Makefile.global and various other places) was different even
though the ./configure invocations were identical. Eventually, I found
that Makefile.global was the culprit, with some sections like:pkglibdir = $(libdir)
ifeq "$(findstring pgsql, $(pkglibdir))" ""
ifeq "$(findstring postgres, $(pkglibdir))" ""
override pkglibdir := $(pkglibdir)/postgresql
endif
endif
Right. See commit 984b0b4df83f and its discussion at [1]/messages/by-id/Pine.LNX.4.21.0010042208350.934-100000@peter.localdomain. The working
theory (as I remember it) is that if the installation is already
postgres-specific (because the user has made it so by having "pgsql" or
"postgres" in the argument to --prefix), then we don't need to add
another level of postgres-specificity to it; but otherwise we add
"/postgresql" to the path so that the resulting files don't end up mixed
with files installed by unrelated packages.
[1]: /messages/by-id/Pine.LNX.4.21.0010042208350.934-100000@peter.localdomain
--
Álvaro Herrera Breisgau, Deutschland — https://www.EnterpriseDB.com/
Hi,
On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote:
On 2022-Jul-15, Andy Bailey wrote:
After some investigation, I found that the value of pkglibdir (as used
by src/Makefile.global and various other places) was different even
though the ./configure invocations were identical. Eventually, I found
that Makefile.global was the culprit, with some sections like:pkglibdir = $(libdir)
ifeq "$(findstring pgsql, $(pkglibdir))" ""
ifeq "$(findstring postgres, $(pkglibdir))" ""
override pkglibdir := $(pkglibdir)/postgresql
endif
endifRight. See commit 984b0b4df83f and its discussion at [1]. The working
theory (as I remember it) is that if the installation is already
postgres-specific (because the user has made it so by having "pgsql" or
"postgres" in the argument to --prefix), then we don't need to add
another level of postgres-specificity to it; but otherwise we add
"/postgresql" to the path so that the resulting files don't end up mixed
with files installed by unrelated packages.
I think the current approach isn't great.
E.g., putting extension .so's into the same directories as "normal shared
libraries", which typically is either in the library search path via system
configuration, or in the library search path due to rpath, strikes me as quite
wrongheaded. They're not libraries that should be linked to, so they shouldn't
be in the library dir directly.
Nor do we really allow the user to opt in/out of us changing the
user-specified install locations.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote:
Right. See commit 984b0b4df83f and its discussion at [1]. The working
theory (as I remember it) is that if the installation is already
postgres-specific (because the user has made it so by having "pgsql" or
"postgres" in the argument to --prefix), then we don't need to add
another level of postgres-specificity to it; but otherwise we add
"/postgresql" to the path so that the resulting files don't end up mixed
with files installed by unrelated packages.
I think the current approach isn't great.
E.g., putting extension .so's into the same directories as "normal shared
libraries", which typically is either in the library search path via system
configuration, or in the library search path due to rpath, strikes me as quite
wrongheaded.
Well, we don't, with a normal not-PG-specific install path.
What I get with --prefix=/home/tgl/testversion is
$ ls ~/testversion/lib
libecpg.a libecpg_compat.so@ libpgtypes.so@ libpq.so.5@
libecpg.so@ libecpg_compat.so.3@ libpgtypes.so.3@ libpq.so.5.5*
libecpg.so.6@ libecpg_compat.so.3.4* libpgtypes.so.3.3* postgresql/
libecpg.so.6.4* libpgport.a libpq.a
libecpg_compat.a libpgtypes.a libpq.so@
$ ls ~/testversion/lib/postgresql
ascii_and_mic.so* pgxs/ utf8_and_gb18030.so*
cyrillic_and_mic.so* plperl.so* utf8_and_gbk.so*
dict_snowball.so* plpgsql.so* utf8_and_iso8859.so*
euc2004_sjis2004.so* utf8_and_ascii.so* utf8_and_iso8859_1.so*
euc_cn_and_mic.so* utf8_and_big5.so* utf8_and_johab.so*
euc_jp_and_sjis.so* utf8_and_cyrillic.so* utf8_and_sjis.so*
euc_kr_and_mic.so* utf8_and_euc2004.so* utf8_and_sjis2004.so*
euc_tw_and_big5.so* utf8_and_euc_cn.so* utf8_and_uhc.so*
latin2_and_win1250.so* utf8_and_euc_jp.so* utf8_and_win.so*
latin_and_mic.so* utf8_and_euc_kr.so*
libpqwalreceiver.so* utf8_and_euc_tw.so*
which seems perfectly sane from here. The issue is that if the
install directory path already has "postgres" somewhere in it,
you get a different layout where the PG-specific subdir is smashed
together with the user libraries.
I tend to agree with the OP that that case isn't great, but we've
had it like this for so long that I'm not sure we can change it.
OTOH, the issue probably only affects PG developers, so maybe
we could?
Nor do we really allow the user to opt in/out of us changing the
user-specified install locations.
Yeah, you'd have to resort to editing Makefile.global after
configuring, which is annoying.
regards, tom lane
Hi,
On 2022-07-15 19:22:12 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On 2022-07-15 20:38:17 +0200, Alvaro Herrera wrote:
Right. See commit 984b0b4df83f and its discussion at [1]. The working
theory (as I remember it) is that if the installation is already
postgres-specific (because the user has made it so by having "pgsql" or
"postgres" in the argument to --prefix), then we don't need to add
another level of postgres-specificity to it; but otherwise we add
"/postgresql" to the path so that the resulting files don't end up mixed
with files installed by unrelated packages.I think the current approach isn't great.
E.g., putting extension .so's into the same directories as "normal shared
libraries", which typically is either in the library search path via system
configuration, or in the library search path due to rpath, strikes me as quite
wrongheaded.Well, we don't, with a normal not-PG-specific install path.
Right - I was commenting on the "postgres"/"pgsql" specific logic.
which seems perfectly sane from here. The issue is that if the
install directory path already has "postgres" somewhere in it,
you get a different layout where the PG-specific subdir is smashed
together with the user libraries.I tend to agree with the OP that that case isn't great, but we've
had it like this for so long that I'm not sure we can change it.
OTOH, the issue probably only affects PG developers, so maybe
we could?
I'm not sure it just devs - I think several distribution install things into
per-major-version paths containing postgres or pgsql. But I think they also
tend to patch postgres to avoid some of the Makefile.global magic. So I
suspect it'd not be too hard for them to adjust.
Greetings,
Andres Freund