[PATCH] jit: fix build with LLVM-21
(please cc: me on replies)
Hello!
I tried building against LLVM-21 and noticed that a function for symbol
lookup was renamed (without semantic changes), breaking the LLVM JIT.
The following patch fixes this by adding a version guard. It applies equally
to both master and 17.6. Passes the test suite and verified on 17.6 with the
jit example from the documentation.
cheers,
Holger
Attachments:
llvm21.patchtext/x-patch; charset=UTF-8; name=llvm21.patchDownload
From 932b8f540f157ce9bcbae9ff618d507e9229778e Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Holger=20Hoffst=C3=A4tte?= <holger@applied-asynchrony.com>
Date: Mon, 8 Sep 2025 14:57:32 +0200
Subject: [PATCH] jit: fix build with LLVM-21
LLVM-21 renamed llvm::GlobalValue::getGUID() to
getGUIDAssumingExternalLinkage(), so add a version guard.
---
src/backend/jit/llvm/llvmjit_inline.cpp | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/jit/llvm/llvmjit_inline.cpp b/src/backend/jit/llvm/llvmjit_inline.cpp
index 2764c3bbe2f..39a19ec6a46 100644
--- a/src/backend/jit/llvm/llvmjit_inline.cpp
+++ b/src/backend/jit/llvm/llvmjit_inline.cpp
@@ -238,7 +238,11 @@ llvm_build_inline_plan(LLVMContextRef lc, llvm::Module *mod)
llvm_split_symbol_name(symbolName.data(), &cmodname, &cfuncname);
+#if LLVM_VERSION_MAJOR < 21
funcGUID = llvm::GlobalValue::getGUID(cfuncname);
+#else
+ funcGUID = llvm::GlobalValue::getGUIDAssumingExternalLinkage(cfuncname);
+#endif
/* already processed */
if (inlineState.processed)
--
2.51.0
On 08.09.25 15:20, Holger Hoffstätte wrote:
I tried building against LLVM-21 and noticed that a function for symbol
lookup was renamed (without semantic changes), breaking the LLVM JIT.
The following patch fixes this by adding a version guard. It applies
equally
to both master and 17.6. Passes the test suite and verified on 17.6 with
the
jit example from the documentation.
I can confirm that this change seems correct. See [0]https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3b for reference.
[0]: https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3b
https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3b
As a small style request, I would flip the conditional around so that
the new code appears first. I see that we don't do this very
consistently in the existing code, but maybe we can start a new trend. ;-)
In my testing with LLVM 21, I'm getting an additional error:
../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching
constructor for initialization of 'llvm::orc::RTDyldObjectLinkingLayer'
56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
| ^
57 | *unwrap(ES), [] { return
std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true); }));
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3:
note: candidate constructor not viable: no known conversion from
'(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to
'GetMemoryManagerFunction' (aka
'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const
MemoryBuffer &)>') for 2nd argument
58 | RTDyldObjectLinkingLayer(ExecutionSession &ES,
| ^
59 | GetMemoryManagerFunction
GetMemoryManager);
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16:
note: candidate constructor (the implicit copy constructor) not viable:
requires 1 argument, but 2 were provided
37 | class LLVM_ABI RTDyldObjectLinkingLayer
| ^~~~~~~~~~~~~~~~~~~~~~~~
I gather you're not seeing that?
On 2025-09-12 08:36, Peter Eisentraut wrote:
On 08.09.25 15:20, Holger Hoffstätte wrote:
I tried building against LLVM-21 and noticed that a function for symbol
lookup was renamed (without semantic changes), breaking the LLVM JIT.
The following patch fixes this by adding a version guard. It applies equally
to both master and 17.6. Passes the test suite and verified on 17.6 with the
jit example from the documentation.I can confirm that this change seems correct. See [0] for reference.
Excellent! Thanks for taking a look. This was my first post to pgsql-hackers
and I wasn't sure if I had done something wrong.
[0]:
https://github.com/llvm/llvm-project/commit/d3d856ad84698fa4ec66177d00558b2f5b438d3bAs a small style request, I would flip the conditional around so
that the new code appears first. I see that we don't do this very
consistently in the existing code, but maybe we can start a new
trend. ;-)
I knew this would come up since I pondered the same thing :D
As you said, the existing code is not consistent, but I can switch this
to let the new code appear first, if you prefer.
A new patch is attached.
In my testing with LLVM 21, I'm getting an additional error:
../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching constructor for initialization of 'llvm::orc::RTDyldObjectLinkingLayer'
56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
| ^
57 | *unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true); }));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3: note: candidate constructor not viable: no known conversion from '(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to 'GetMemoryManagerFunction' (aka 'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const MemoryBuffer &)>') for 2nd argument
58 | RTDyldObjectLinkingLayer(ExecutionSession &ES,
| ^
59 | GetMemoryManagerFunction GetMemoryManager);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
37 | class LLVM_ABI RTDyldObjectLinkingLayer
| ^~~~~~~~~~~~~~~~~~~~~~~~I gather you're not seeing that?
I do not - I dropped the patch into my Gentoo package build for 17.6 and
it has been working fine when building with gcc-15.2 and the jit part
with clang-21.1.0. I also just rebuilt everything with clang-21.1.1 (no gcc)
and also do not see the problem. This is on amd64 Linux.
The following commit seems to be involved:
https://github.com/postgres/postgres/commit/9044fc1d45a0212fd123bd8f364eac058f60fed7
which is also in my 17.6 tree. I verified that the new
SectionMemoryManager.cpp is compiled without error.
Since you're using homebrew I guess this is triggered by the
__aarch64__ guard in src/include/jit/llvmjit_backport.h.
Unfortunately this is as far as I can help with this.
Please let me know if there is anything else I can do.
cheers
Holger
Attachments:
llvm-21-v2.patchtext/x-patch; charset=UTF-8; name=llvm-21-v2.patchDownload
From 699a5aa8f9438c902ac5ad063525193ddf0ebb59 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Holger=20Hoffst=C3=A4tte?= <holger@applied-asynchrony.com>
Date: Fri, 12 Sep 2025 09:40:31 +0200
Subject: [PATCH] jit: fix build with LLVM-21
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
LLVM-21 renamed llvm::GlobalValue::getGUID() to
getGUIDAssumingExternalLinkage(), so add a version guard.
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
---
src/backend/jit/llvm/llvmjit_inline.cpp | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/src/backend/jit/llvm/llvmjit_inline.cpp b/src/backend/jit/llvm/llvmjit_inline.cpp
index 2764c3bbe2f..51b32cd9f94 100644
--- a/src/backend/jit/llvm/llvmjit_inline.cpp
+++ b/src/backend/jit/llvm/llvmjit_inline.cpp
@@ -238,7 +238,11 @@ llvm_build_inline_plan(LLVMContextRef lc, llvm::Module *mod)
llvm_split_symbol_name(symbolName.data(), &cmodname, &cfuncname);
+#if LLVM_VERSION_MAJOR >= 21
+ funcGUID = llvm::GlobalValue::getGUIDAssumingExternalLinkage(cfuncname);
+#else
funcGUID = llvm::GlobalValue::getGUID(cfuncname);
+#endif
/* already processed */
if (inlineState.processed)
--
2.51.0
On 2025-09-12 09:47, Holger Hoffstätte wrote:
On 2025-09-12 08:36, Peter Eisentraut wrote:
In my testing with LLVM 21, I'm getting an additional error:
../src/backend/jit/llvm/llvmjit_wrap.cpp:56:18: error: no matching constructor for initialization of 'llvm::orc::RTDyldObjectLinkingLayer'
56 | return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
| ^
57 | *unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true); }));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:58:3: note: candidate constructor not viable: no known conversion from '(lambda at ../src/backend/jit/llvm/llvmjit_wrap.cpp:57:16)' to 'GetMemoryManagerFunction' (aka 'unique_function<std::unique_ptr<RuntimeDyld::MemoryManager> (const MemoryBuffer &)>') for 2nd argument
58 | RTDyldObjectLinkingLayer(ExecutionSession &ES,
| ^
59 | GetMemoryManagerFunction GetMemoryManager);
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/opt/homebrew/Cellar/llvm/21.1.0/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h:37:16: note: candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 2 were provided
37 | class LLVM_ABI RTDyldObjectLinkingLayer
| ^~~~~~~~~~~~~~~~~~~~~~~~I gather you're not seeing that?
I temporarily removed the __arch64__ guard and can reproduce the above error,
using either gcc-15 or clang-21. It seems this llvm monkey patch backport thing
is affected by the following commit in llvm-21:
https://github.com/llvm/llvm-project/commit/cd585864c0bbbd74ed2a2b1ccc191eed4d1c8f90
Trying to figure out how to adapt the code..
chers
Holger
On 2025-09-12 14:47, Holger Hoffstätte wrote:
<snip>
I temporarily removed the __arch64__ guard and can reproduce the above error,
using either gcc-15 or clang-21. It seems this llvm monkey patch backport thing
is affected by the following commit in llvm-21:
https://github.com/llvm/llvm-project/commit/cd585864c0bbbd74ed2a2b1ccc191eed4d1c8f90Trying to figure out how to adapt the code..
Try the attached patch on your homebrew setup.
This compiles and passes "make check", though I do not think it runs any jit tests.
So intead I dropped this into my 17.6, disabled the __arch64__ guard to make sure
I get the backport class, rebuilt and ran the jit on/off example from the docs.
This showed the expected performance difference with jit=on vs. off.
cheers
Holger
Attachments:
llvm-21-aarch64.patchtext/x-patch; charset=UTF-8; name=llvm-21-aarch64.patchDownload
From da8c15ef4f8dbfe931fe2700d849f49078bfca97 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Holger=20Hoffst=C3=A4tte?= <holger@applied-asynchrony.com>
Date: Fri, 12 Sep 2025 16:01:15 +0200
Subject: [PATCH] jit: update aarch64-only use of SectionMemoryManager for
LLVM-21
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
LLVM-21 changed the invocation of RTDyldObjectLinkingLayer, breaking
compilation with the backported SectionMemoryManager for aarch64.
Introduce a version guard for the newly passed parameter.
Signed-off-by: Holger Hoffstätte <holger@applied-asynchrony.com>
---
src/backend/jit/llvm/llvmjit_wrap.cpp | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/src/backend/jit/llvm/llvmjit_wrap.cpp b/src/backend/jit/llvm/llvmjit_wrap.cpp
index da850d67ab6..d98e3d0d900 100644
--- a/src/backend/jit/llvm/llvmjit_wrap.cpp
+++ b/src/backend/jit/llvm/llvmjit_wrap.cpp
@@ -53,7 +53,14 @@ DEFINE_SIMPLE_CONVERSION_FUNCTIONS(llvm::orc::ObjectLayer, LLVMOrcObjectLayerRef
LLVMOrcObjectLayerRef
LLVMOrcCreateRTDyldObjectLinkingLayerWithSafeSectionMemoryManager(LLVMOrcExecutionSessionRef ES)
{
+#if LLVM_VERSION_MAJOR >= 21
+ return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
+ *unwrap(ES), [](const llvm::MemoryBuffer &) {
+ return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true);
+ }));
+#else
return wrap(new llvm::orc::RTDyldObjectLinkingLayer(
*unwrap(ES), [] { return std::make_unique<llvm::backport::SectionMemoryManager>(nullptr, true); }));
+#endif
}
#endif
--
2.51.0
On 12.09.25 09:47, Holger Hoffstätte wrote:
As a small style request, I would flip the conditional around so
that the new code appears first. I see that we don't do this very
consistently in the existing code, but maybe we can start a new
trend. ;-)I knew this would come up since I pondered the same thing :D
As you said, the existing code is not consistent, but I can switch this
to let the new code appear first, if you prefer.
A new patch is attached.
I have committed this one. For the remaining issue on arm, let's wait
for some additional reviews.
Hi,
I've tested the patch: it fixes the compilation on arm and I'm
correctly linked to llvm21:
ldd /var/lib/postgresql/.local/lib/llvmjit.so | grep libLLVM
libLLVM.so.21.1 => /lib/aarch64-linux-gnu/libLLVM.so.21.1
(0x0000ec34112d0000)
Testing the query that used to trigger the jit segfault, the llvm
backport is working as expected.
pgbench -i --partitions=256
psql options=-cjit_above_cost=0 -c 'SELECT count(bid) from pgbench_accounts;'
count
--------
100000
As a comparison, if I forcefully disable
USE_LLVM_BACKPORT_SECTION_MEMORY_MANAGER, I have the expected crash as
the issue is still present in LLVM (the fix would be to switch to
JITLink).
psql options=-cjit_above_cost=0 -c 'SELECT count(bid) from pgbench_accounts;'
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
connection to server was lost
On 2025-09-12 16:22, Holger Hoffstätte wrote:
On 2025-09-12 14:47, Holger Hoffstätte wrote:
<snip>I temporarily removed the __arch64__ guard and can reproduce the above error,
using either gcc-15 or clang-21. It seems this llvm monkey patch backport thing
is affected by the following commit in llvm-21:
https://github.com/llvm/llvm-project/commit/cd585864c0bbbd74ed2a2b1ccc191eed4d1c8f90Trying to figure out how to adapt the code..
Try the attached patch on your homebrew setup.
This compiles and passes "make check", though I do not think it runs any jit tests.
So intead I dropped this into my 17.6, disabled the __arch64__ guard to make sure
I get the backport class, rebuilt and ran the jit on/off example from the docs.
This showed the expected performance difference with jit=on vs. off.
Any news on the aarch64 front? The patch I sent in
/messages/by-id/44718204-61a5-91b1-5574-154ff7a019f6@applied-asynchrony.com
seemed to work for Anthonin Bonnefoy:
/messages/by-id/CAO6_XqqbrO3O48f+XM0fp6P8ha61sZqYPuri++P0+KMVM0xcdw@mail.gmail.com
Just trying to make sure this doesn't fall into the couch crack.
thanks,
Holger
Homebrew is now installing LLVM21 by default, so compiling with Homebrew's
default LLVM now fails without this patch.
Just trying to make sure this doesn't fall into the couch crack.
I don't see a commitfest[1]https://commitfest.postgresql.org/ entry for this patch. You should create one to
keep track of the patch.
On Mon, Nov 17, 2025 at 9:48 PM Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:
Homebrew is now installing LLVM21 by default, so compiling with Homebrew's default LLVM now fails without this patch.
Ack, testing...
On 2025-11-17 12:11, Thomas Munro wrote:
On Mon, Nov 17, 2025 at 9:48 PM Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:Homebrew is now installing LLVM21 by default, so compiling with Homebrew's default LLVM now fails without this patch.
Ack, testing...
I made an account on commitfest but cannot log in yet due
to some "cooloff period" (presumably to ward off spammers).
Would be nice if you can get this sorted.
thanks
Holger
On 17 Nov 2025, at 12:31, Holger Hoffstätte <holger@applied-asynchrony.com> wrote:
I made an account on commitfest but cannot log in yet due
to some "cooloff period" (presumably to ward off spammers).
That's indeed the reason (and sadly we need it).
Would be nice if you can get this sorted.
Posting a request to pgsql-www@ is a good way to make sure the team that can
expedite processing doesn't miss it among all active threads here on -hackers.
--
Daniel Gustafsson
On Tue, Nov 18, 2025 at 12:11 AM Thomas Munro <thomas.munro@gmail.com> wrote:
On Mon, Nov 17, 2025 at 9:48 PM Anthonin Bonnefoy
<anthonin.bonnefoy@datadoghq.com> wrote:Homebrew is now installing LLVM21 by default, so compiling with Homebrew's default LLVM now fails without this patch.
Ack, testing...
Looks good so far. Will try to push this tomorrow.
I got distracted trying to figure out why LLVM 22 is segfaulting in
llvm::orc::ExecutionSession::IL_emit(). It's not out ... yet. I was
worried that might be a clue about problems with the backported
SectionMemoryManager nonsense, but (un)fortunately it also still
crashes with the JITLink patch[1]/messages/by-id/CA+hUKGJBJx4fDGLv8zUtmsmg16Swry7DJbMr2_GNZcd6sgE0rg@mail.gmail.com. Will need to chase this down...
[1]: /messages/by-id/CA+hUKGJBJx4fDGLv8zUtmsmg16Swry7DJbMr2_GNZcd6sgE0rg@mail.gmail.com
On Tue, Nov 18, 2025 at 4:06 AM Thomas Munro <thomas.munro@gmail.com> wrote:
Looks good so far. Will try to push this tomorrow.
Pushed. Thanks Holger and Anthonin.
I got distracted trying to figure out why LLVM 22 is segfaulting in
llvm::orc::ExecutionSession::IL_emit(). It's not out ... yet. I was
worried that might be a clue about problems with the backported
SectionMemoryManager nonsense, but (un)fortunately it also still
crashes with the JITLink patch[1]. Will need to chase this down...
Got a bit further with LLVM 22, let's see... (clues/ideas welcome).