Setting rpath on llvmjit.so?
Currently, we don't set rpath for llvmjit.so to include `llvm-config --
libdir`. That means you need to keep LD_LIBRARY_PATH set properly to
use any non-system installation of LLVM.
This isn't unique to LLVM (we only set rpath to the postgres $libdir),
so I thought it might be project policy. But I think it will be worse
for LLVM (and perhaps ICU) where people are more likely to want to run
specific versions. Comments?
Included a simple patch, but it could use some review from someone who
knows more about portable linking and our build system.
Regards,
Jeff Davis
PS: I also noticed that a autoconf run on master generates some diffs
in configure.
Attachments:
llvm-rpath.difftext/x-patch; charset=UTF-8; name=llvm-rpath.diffDownload
diff --git a/config/llvm.m4 b/config/llvm.m4
index e25ffec..582ba51 100644
--- a/config/llvm.m4
+++ b/config/llvm.m4
@@ -90,6 +90,7 @@ AC_DEFUN([PGAC_LLVM_SUPPORT],
done
LLVM_BINPATH=`$LLVM_CONFIG --bindir`
+ LLVM_LIBPATH=`$LLVM_CONFIG --libdir`
# Check which functionality is present
SAVE_CPPFLAGS="$CPPFLAGS"
@@ -104,5 +105,6 @@ AC_DEFUN([PGAC_LLVM_SUPPORT],
AC_SUBST(LLVM_CFLAGS)
AC_SUBST(LLVM_CXXFLAGS)
AC_SUBST(LLVM_BINPATH)
+ AC_SUBST(LLVM_LIBPATH)
])# PGAC_LLVM_SUPPORT
diff --git a/configure b/configure
index a30d7be..b070658 100755
--- a/configure
+++ b/configure
@@ -730,6 +730,7 @@ CPP
BITCODE_CXXFLAGS
BITCODE_CFLAGS
CFLAGS_VECTOR
+LLVM_LIBPATH
LLVM_BINPATH
LLVM_CXXFLAGS
LLVM_CFLAGS
@@ -5005,6 +5006,7 @@ fi
done
LLVM_BINPATH=`$LLVM_CONFIG --bindir`
+ LLVM_LIBPATH=`$LLVM_CONFIG --libdir`
# Check which functionality is present
SAVE_CPPFLAGS="$CPPFLAGS"
@@ -5065,6 +5067,7 @@ _ACEOF
+
fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 2529202..e3cdc13 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -229,6 +229,7 @@ PTHREAD_LIBS = @PTHREAD_LIBS@
LLVM_CONFIG = @LLVM_CONFIG@
LLVM_BINPATH = @LLVM_BINPATH@
+LLVM_LIBPATH = @LLVM_LIBPATH@
CLANG = @CLANG@
BITCODE_CFLAGS = @BITCODE_CFLAGS@
BITCODE_CXXFLAGS = @BITCODE_CXXFLAGS@
diff --git a/src/backend/jit/llvm/Makefile b/src/backend/jit/llvm/Makefile
index e2db4ce..009ff47 100644
--- a/src/backend/jit/llvm/Makefile
+++ b/src/backend/jit/llvm/Makefile
@@ -39,6 +39,8 @@ OBJS=$(WIN32RES)
OBJS += llvmjit.o llvmjit_error.o llvmjit_inline.o llvmjit_wrap.o
# Code generation
OBJS += llvmjit_expr.o llvmjit_deform.o
+# Add LLVM library path to rpath
+rpathdir := $(LLVM_LIBPATH):$(rpathdir)
all: all-shared-lib llvmjit_types.bc
Hi,
On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
Currently, we don't set rpath for llvmjit.so to include `llvm-config --
libdir`. That means you need to keep LD_LIBRARY_PATH set properly to
use any non-system installation of LLVM.
Or pass in the appropriate flags to configure. My configure invoking
script has: LDFLAGS='-Wl,-rpath,/home/andres/build/llvm/6/opt/install/lib' etc
in it.
This isn't unique to LLVM (we only set rpath to the postgres $libdir),
so I thought it might be project policy. But I think it will be worse
for LLVM (and perhaps ICU) where people are more likely to want to run
specific versions. Comments?
I'm not sure we want to do that - and if so to which libraries. I tend
towards thinking that if we do that, we should do it uniformly, not just
for LLVM. At the very least it'd have to respect --disable-rpath.
PS: I also noticed that a autoconf run on master generates some diffs
in configure.
I only see one copyright year related diff. Perhaps you're using a
patched distribution autoconf?
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
PS: I also noticed that a autoconf run on master generates some diffs
in configure.
I only see one copyright year related diff. Perhaps you're using a
patched distribution autoconf?
The project policy is to use exactly the GNU distribution of autoconf.
Distros tend to hack it around (especially since it's been so long
since their last release :-(), so you more or less have to install a
private copy to get matching results. Fortunately it's pretty
trivial to build/install from source.
regards, tom lane
On April 14, 2018 1:56:08 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Andres Freund <andres@anarazel.de> writes:
On 2018-04-14 12:32:35 -0700, Jeff Davis wrote:
PS: I also noticed that a autoconf run on master generates some
diffs
in configure.
I only see one copyright year related diff. Perhaps you're using a
patched distribution autoconf?The project policy is to use exactly the GNU distribution of autoconf.
Distros tend to hack it around (especially since it's been so long
since their last release :-(), so you more or less have to install a
private copy to get matching results. Fortunately it's pretty
trivial to build/install from source.
Fwiw, I see one copyright year related diff with unmodified upstream autoconf.
Andres
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Andres Freund <andres@anarazel.de> writes:
On April 14, 2018 1:56:08 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
The project policy is to use exactly the GNU distribution of autoconf.
Fwiw, I see one copyright year related diff with unmodified upstream autoconf.
Really? Where did you get autoconf from? My archived copy of
autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
dates of (mostly) 2012-04-24. That matches the file dates at
https://ftp.gnu.org/gnu/autoconf/
so I've not actually pulled down fresh bits to compare, but ...
regards, tom lane
On 2018-04-14 17:10:21 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On April 14, 2018 1:56:08 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
The project policy is to use exactly the GNU distribution of autoconf.
Fwiw, I see one copyright year related diff with unmodified upstream autoconf.
Really? Where did you get autoconf from? My archived copy of
autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
dates of (mostly) 2012-04-24. That matches the file dates at
https://ftp.gnu.org/gnu/autoconf/
so I've not actually pulled down fresh bits to compare, but ...
The v2.69 git tag. So I guess that's set somewhere in the tarball
building. Hrmpf.
The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...
Greetings,
Andres Freund
2018-04-15 6:13 GMT+09:00 Andres Freund <andres@anarazel.de>:
On 2018-04-14 17:10:21 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On April 14, 2018 1:56:08 PM PDT, Tom Lane <tgl@sss.pgh.pa.us> wrote:
The project policy is to use exactly the GNU distribution of autoconf.
Fwiw, I see one copyright year related diff with unmodified upstream
autoconf.
Really? Where did you get autoconf from? My archived copy of
autoconf-2.69.tar.gz, fetched 2013-11-15, has internal file
dates of (mostly) 2012-04-24. That matches the file dates at
https://ftp.gnu.org/gnu/autoconf/
so I've not actually pulled down fresh bits to compare, but ...The v2.69 git tag. So I guess that's set somewhere in the tarball
building. Hrmpf.The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...
My cmake branch still working and I supporting stable postgres releases:
https://github.com/stalkerg/postgres_cmake
Also, I looked into Meson but unfortunately some features still not exist
for building postgres.
Anyway, my branch exist, cmake working fine maybe without minor features
like generating documentation.
My cmake version also have minor different behaviour but it because cmake
not build your project directly, it more like project generator (same as
automake and meson).
We can start conversation about it again, I am open for this.
Regards
On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund <andres@anarazel.de> wrote:
The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...
I don't necessarily see that as a reason to move. If we're hitting
bugs in autoconf that are causing problems, and we can't get them
fixed in the upstream, that's a reason. But switching build systems
won't be frictionless, so I'm not in favor of it unless there's a
clear payoff.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Robert Haas <robertmhaas@gmail.com> writes:
... But switching build systems
won't be frictionless,
I think we have a nominee for Understatement of the Year.
regards, tom lane
On 2018-04-17 15:56:14 -0400, Robert Haas wrote:
On Sat, Apr 14, 2018 at 5:13 PM, Andres Freund <andres@anarazel.de> wrote:
The half unmaintainedness of autoconf (no release in five years counts
as that imo), sure makes it look like a good idea to move on to cmake or
such at some point...I don't necessarily see that as a reason to move. If we're hitting
bugs in autoconf that are causing problems, and we can't get them
fixed in the upstream, that's a reason. But switching build systems
won't be frictionless, so I'm not in favor of it unless there's a
clear payoff.
I'd not advocate for this solely based on the age of autoconf. But the
separate windows buildsystem which makes it very hard to build
extensions separately is a good reason on its own. As is the fact that
recursive make as we're using it has significant issues. Both of those
would be avoided by using cmake or such.
Greetings,
Andres Freund
Hi,
On 2018-04-16 11:26:14 +0900, Yuriy Zhuravlev wrote:
My cmake branch still working and I supporting stable postgres releases:
https://github.com/stalkerg/postgres_cmake
Anyway, my branch exist, cmake working fine maybe without minor features
like generating documentation.
We can start conversation about it again, I am open for this.
Well, before it does everything, there's little point in reviewing
whether it's mergeable or not.
I still think cmake is the least unreasonable path going forward.
Greetings,
Andres Freund
Well, before it does everything, there's little point in reviewing
whether it's mergeable or not.
For this significant case, it's not working as you expect.
First, Postgres community should find consensus about migration to CMake
(or alternative). Now, this project too huge to work on it without sure in
importance.
Second, a few main contributors should start helping to fix bugs and deep
into architecture. It's very important because build system very tightly
bound with all source code and one man can't know all rarely cases.
Moreover, it's needed to understand some restriction CMake (and another
project generators) and find consensus in the community about it. For
example why we can't build contrib modules independent on main postgres
with CMake.
Also, you should understand what you can't keep the whole feature set and
behaviors in the new build system, postgres too big for this now (probably
you will have many new features but you will lose too). It's why main
contributors should know new build system to find solutions and consensus
for decisions.
We can open a new thread for discussion about the first question, the
second question should be later.
On 4/17/18 16:14, Andres Freund wrote:
I still think cmake is the least unreasonable path going forward.
I would rather try to make Meson work and if needed add features back
into Meson.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
On Tue, Apr 17, 2018 at 4:13 PM, Andres Freund <andres@anarazel.de> wrote:
I'd not advocate for this solely based on the age of autoconf. But the
separate windows buildsystem which makes it very hard to build
extensions separately is a good reason on its own. As is the fact that
recursive make as we're using it has significant issues. Both of those
would be avoided by using cmake or such.
How much of the pain building extensions on Windows comes from our
build system and how much of it comes from the fact that it's Windows?
My guess would have been that the latter is by far the bigger issue,
but maybe I'm wrong.
There is really nothing keeping us from removing or reducing the use
of recursive make without switching to cmake. And it's probably not
even that much work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
On 4/17/18 16:14, Andres Freund wrote:
I still think cmake is the least unreasonable path going forward.
I would rather try to make Meson work and if needed add features back
into Meson.
I'm wondering whether that will result in expending a lot of effort
to move from a poorly-supported build system to a different
poorly-supported build system. As a project, we have an astonishingly
poor track record for choosing outside projects to rely on :-(.
One thing I particularly dislike about Meson, after reading its wikipedia
entry for a moment, is that it doesn't use "make" as the backend build
system. That means two major tooling changes not one, and double the
opportunity to find ourselves stuck with unsupported infrastructure.
I also politely decline the offer to be forced to use XCode on mac.
regards, tom lane
Current autoconf system not working on Windows at all, what we talk about?
On Wed, 18 Apr 2018, 23:57 Robert Haas, <robertmhaas@gmail.com> wrote:
Show quoted text
On Tue, Apr 17, 2018 at 4:13 PM, Andres Freund <andres@anarazel.de> wrote:
I'd not advocate for this solely based on the age of autoconf. But the
separate windows buildsystem which makes it very hard to build
extensions separately is a good reason on its own. As is the fact that
recursive make as we're using it has significant issues. Both of those
would be avoided by using cmake or such.How much of the pain building extensions on Windows comes from our
build system and how much of it comes from the fact that it's Windows?
My guess would have been that the latter is by far the bigger issue,
but maybe I'm wrong.There is really nothing keeping us from removing or reducing the use
of recursive make without switching to cmake. And it's probably not
even that much work.--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
On 2018-04-18 15:54:59 +0000, Yuriy Zhuravlev wrote:
Current autoconf system not working on Windows at all, what we talk about?
We generate windows project files. See src/tools/msvc/
- Andres
I also politely decline the offer to be forced to use XCode on mac.
Why? We supporting MSVC and not nmake, what difference with xcode? Also,
it's just extra benefit from cmake/meson.
I talked about autoconf build system, /tools/msvc it's extra home build
system.
On Thu, 19 Apr 2018, 00:58 Andres Freund, <andres@anarazel.de> wrote:
Show quoted text
On 2018-04-18 15:54:59 +0000, Yuriy Zhuravlev wrote:
Current autoconf system not working on Windows at all, what we talk
about?
We generate windows project files. See src/tools/msvc/
- Andres
On Apr 18, 2018, at 8:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
I'm wondering whether that will result in expending a lot of effort to move from a poorly-supported build system to a different poorly-supported build system.
I’m not sure whether the former is autoconf/make or cmake, but count me as another vote for cmake over meson. CMake has direct support from Microsoft (Visual Studio 2017 can auto-ingest CMake-based files as soon as they’re checked out) and JetBrains CLion and is in use by LLVM, KDE, Qt, Blender, libpng, cURL, LAPACK, MySQL/MariaDB, OpenCV, SDL, the Dolphin Gamecube emulator etc.
So the tooling around people using it is there and it’s used by some very large and mature projects covering a variety of domains: I don’t know if it’s “poorly supported” (the documentation leaves something to be desired), but if CMake has trouble down the line a lot of very important projects will be in trouble. It seems like a pretty safe bet given the sheer inertia of the list above.
--
Jason Petersen
Software Engineer | Citus Data
303.736.9255
jason@citusdata.com
On Thu, Apr 26, 2018 at 01:16:09PM -0600, Jason Petersen wrote:
So the tooling around people using it is there and it’s used by some
very large and mature projects covering a variety of domains: I don’t
know if it’s “poorly supported” (the documentation leaves something to
be desired), but if CMake has trouble down the line a lot of very
important projects will be in trouble. It seems like a pretty safe bet
given the sheer inertia of the list above.
Yes, those would be arguments pushing in favor of cmake. Yuriy has
mentioned me a couple of times that when he worked on the cmake
integration on Windows he used heavily VS because that was quite
friendly on Windows and their was a good integration work.
--
Michael