JIT on Windows with Postgres 12.1

Hello.

If I understand it well, JIT is enabled only for builds with "--with-llvm"
flag. Seems EDB Windows PG 12.1 installer is not build with llvm (and jit)
support.

I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

út 11. 2. 2020 v 22:41 odesílatel Catch All <catchall@lightn.org> napsal:

Hi,

On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:

I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

It's not currently supported by the windows buildsystem. I don't
personally have the expertise to change that (I've not worked on windows
after ~2003). So somebody would have to step in to make that work,
first.

Greetings,

Andres Freund

Andres Freund <andres@anarazel.de> writes:

On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:

I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

It's not currently supported by the windows buildsystem. I don't
personally have the expertise to change that (I've not worked on windows
after ~2003). So somebody would have to step in to make that work,
first.

First you would need either a Windows-native LLVM, or a JIT provider
based on Microsoft's compiler. I don't know if the former exists
(it would not be our job to create it). I'm not sure if the latter
is possible, but for sure it'd be a large task if it is possible.

regards, tom lane

Here is a link to build LLVM on windows

http://llvm.org/docs/GettingStartedVS.html

On Wed, Feb 12, 2020 at 2:55 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

Hi,

On 2020-02-12 14:54:56 -0500, Tom Lane wrote:

Andres Freund <andres@anarazel.de> writes:

On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote:

I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled.

It's not currently supported by the windows buildsystem. I don't
personally have the expertise to change that (I've not worked on windows
after ~2003). So somebody would have to step in to make that work,
first.

First you would need either a Windows-native LLVM, or a JIT provider
based on Microsoft's compiler. I don't know if the former exists
(it would not be our job to create it).

Yes, LLVM would work on windows. I'd not even be surprised if one could
make it work on windows already when using a mingw based build, instead
of msvc.

I'm not sure if the latter is possible, but for sure it'd be a large
task if it is possible.

Yea, it'd be a lot of work to add a second provider. No iea if there's
any support for it around visual studio - but I also see no point going
there.

Greetings,

Andres Freund

On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:

Yes, LLVM would work on windows. I'd not even be surprised if one could
make it work on windows already when using a mingw based build, instead
of msvc.

For MSVC, assuming that you have the proper dependencies for JIT
compilation and LLVM available, and just for the reference if anybody
is interesting in making that happen (Tom and Andres are aware of that
already), one would need to do roughly an update of src/tools/msvc/ to
add a new option path to LLVM, and update install-windows.sgml to add
the new optional dependency. Note that including LLVM in the builds
means adding conditionally the files part of src/backend/jit/llvm/,
but one can take for example OpenSSL as a model of how it happens.
That would take care of the build. There may be some
Windows-dependent stuff needed to make JIT work properly on Windows
though.
--
Michael

Hi,

On 2020-02-13 12:25:40 +0900, Michael Paquier wrote:

On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:

Yes, LLVM would work on windows. I'd not even be surprised if one could
make it work on windows already when using a mingw based build, instead
of msvc.

For MSVC, assuming that you have the proper dependencies for JIT
compilation and LLVM available, and just for the reference if anybody
is interesting in making that happen (Tom and Andres are aware of that
already), one would need to do roughly an update of src/tools/msvc/ to
add a new option path to LLVM, and update install-windows.sgml to add
the new optional dependency. Note that including LLVM in the builds
means adding conditionally the files part of src/backend/jit/llvm/,
but one can take for example OpenSSL as a model of how it happens.
That would take care of the build.

That's not really the hard part. That's integrating the generation of
LLVM bitcode files into the buildsystem. As the absolute minimum
llvmjit_types.bc needs to be generated, but to be meaningfully supported
we'd need to generate all the .bc files we're generating on make based
builds.

Greetings,

Andres Freund

On Wed, Feb 12, 2020 at 07:32:08PM -0800, Andres Freund wrote:

That's not really the hard part. That's integrating the generation of
LLVM bitcode files into the buildsystem. As the absolute minimum
llvmjit_types.bc needs to be generated, but to be meaningfully supported
we'd need to generate all the .bc files we're generating on make based
builds.

Oh, I see. That's indeed... Not straight-forward.
--
Michael

On Wed, Feb 12, 2020 at 12:05:11PM -0800, Andres Freund wrote:

Yes, LLVM would work on windows. I'd not even be surprised if one could
make it work on windows already when using a mingw based build, instead
of msvc.

In my mingw environment, postgresql build with option "--with-llvm"
failed with the follwing message. Is it difficult to make PostgreSQL
JIT work on windows using mingw?

--Message1
/mingw64/bin/clang -Wno-ignored-attributes -fno-strict-aliasing
-fwrapv -O2 -I../../../../src/include -I./src/include/port/win32
"-I../../../../src/include/port/win32" -DWIN32_STACK_RLIMIT=4194304
-DBUILDING_DLL -flto=thin -emit-llvm -c -o latch.bc latch.c
latch.c:680:19: error: static_assert expression is not an integral
constant expression
StaticAssertStmt(WSA_INVALID_EVENT == NULL, "");
~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--

--Message2(When commenting latch.c:680 out)
llvmjit_inline.cpp:32:10: fatal error: sys/mman.h: No such file or directory
32 | #include <sys/mman.h>
| ^~~~~~~~~~~~
compilation terminated.
--

--Environment
PostgreSQL:postgresql13.0
Windows:Windows10
MSYS2 packages:mingw-w64-x86_64-gcc base-devel
mingw-w64-x86_64-llvm(llvm11 installed) mingw-w64-x86_64-clang(clang11
installed)
--

--
Yuuki Fujii