Fix race with LLVM and bison.

Started by Maksim.Melnikov19 days ago4 messageshackers
Jump to latest
#1Maksim.Melnikov
m.melnikov@postgrespro.ru

Hi All.

I've found build error in configuration --with-llvm

CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert
--enable-tap-tests --with-openssl --with-icu  --with-llvm

....

make world-bin -j3

....

cubescan.c:9:10: fatal error: 'cubeparse.h' file not found
    9 | #include "cubeparse.h"  /* must be after cubedata.h for YYSTYPE
and NDBOX */

....
segscan.c:9:10: fatal error: 'segparse.h' file not found
    9 | #include "segparse.h"   /* must be after segdata.h for SEG */

The reason is race, that exist between LLVM compilation and bison source
code generation and compilation can occur first.

Ideally LLVM compilation target should depend on header files targets.

The error is difficult to reproduce and I've done simple patch to have
stable reproducing. Fix patch is also attached.

Thanks.

Attachments:

v1-0002-Patch-simplify-reproducing-of-race-with-LLVM-and-.patchtext/x-patch; charset=UTF-8; name=v1-0002-Patch-simplify-reproducing-of-race-with-LLVM-and-.patchDownload+1-1
v1-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patchtext/x-patch; charset=UTF-8; name=v1-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patchDownload+8-1
#2Andres Freund
andres@anarazel.de
In reply to: Maksim.Melnikov (#1)
Re: Fix race with LLVM and bison.

Hi,

On 2026-03-27 14:31:52 +0300, Maksim.Melnikov wrote:

I've found build error in configuration --with-llvm

CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert
--enable-tap-tests --with-openssl --with-icu  --with-llvm

....

make world-bin -j3

....

cubescan.c:9:10: fatal error: 'cubeparse.h' file not found
    9 | #include "cubeparse.h"  /* must be after cubedata.h for YYSTYPE and
NDBOX */

....
segscan.c:9:10: fatal error: 'segparse.h' file not found
    9 | #include "segparse.h"   /* must be after segdata.h for SEG */

The reason is race, that exist between LLVM compilation and bison source
code generation and compilation can occur first.

Ideally LLVM compilation target should depend on header files targets.

The error is difficult to reproduce and I've done simple patch to have
stable reproducing. Fix patch is also attached.

You don't need a sleep to show there's a problem, you can just do
make -C contrib/cube cubescan.bc

We don't have the same issue in the backend, as for backend code each .bc file
depends on the .o file:

src/backend/common.mk:

ifeq ($(with_llvm), yes)
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)
endif

But for some reason I didn't add the same logic to pgxs.mk.

I think we need something like the attached to make the dependencies work.

I'm a bit worried about breaking some extensions if were to backpatch this. So
I'm somewhat inclined to just fix this in master.

Greetings,

Andres Freund

Attachments:

fix-llvm-contrib-pgxs-deps.difftext/x-diff; charset=us-asciiDownload+8-0
#3Maksim.Melnikov
m.melnikov@postgrespro.ru
In reply to: Andres Freund (#2)
Re: Fix race with LLVM and bison.

Hi

On 3/27/26 21:56, Andres Freund wrote:

Hi,

On 2026-03-27 14:31:52 +0300, Maksim.Melnikov wrote:

I've found build error in configuration --with-llvm

CPPFLAGS="-O2" ./configure --enable-debug --enable-cassert
--enable-tap-tests --with-openssl --with-icu  --with-llvm

....

make world-bin -j3

....

cubescan.c:9:10: fatal error: 'cubeparse.h' file not found
    9 | #include "cubeparse.h"  /* must be after cubedata.h for YYSTYPE and
NDBOX */

....
segscan.c:9:10: fatal error: 'segparse.h' file not found
    9 | #include "segparse.h"   /* must be after segdata.h for SEG */

The reason is race, that exist between LLVM compilation and bison source
code generation and compilation can occur first.

Ideally LLVM compilation target should depend on header files targets.

The error is difficult to reproduce and I've done simple patch to have
stable reproducing. Fix patch is also attached.

You don't need a sleep to show there's a problem, you can just do
make -C contrib/cube cubescan.bc

We don't have the same issue in the backend, as for backend code each .bc file
depends on the .o file:

src/backend/common.mk:

ifeq ($(with_llvm), yes)
objfiles.txt: $(patsubst %.o,%.bc, $(OBJS))
$(patsubst %.o,%.bc, $(OBJS)): $(OBJS)
endif

But for some reason I didn't add the same logic to pgxs.mk.

I think we need something like the attached to make the dependencies work.

I'm a bit worried about breaking some extensions if were to backpatch this. So
I'm somewhat inclined to just fix this in master.

Greetings,

Andres Freund

Thanks for attentions, I see your patch is better. Few additional thoughts

1. Now I see that we have two different places to configure backend and
extensions, maybe we should

apply your patch to some common place, for example
src/Makefile.global.in, because it seems common

build logic. How do you think?

2. If you have some doubts about backpatch, I suggest to apply
fix-llvm-contrib-pgxs-deps.diff on master

and patch
v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patch as
backpatch, because this one

correct it only for problematic contribs. How do you think?

Greetings,

Maksim Melnikov

Attachments:

v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patchtext/x-patch; charset=UTF-8; name=v2-0001-Fix-race-with-LLVM-compilation-and-bison-code-gen.patchDownload+8-1
#4Zsolt Parragi
zsolt.parragi@percona.com
In reply to: Andres Freund (#2)
Re: Fix race with LLVM and bison.

I think we need something like the attached to make the dependencies work.

I'm a bit worried about breaking some extensions if were to backpatch this. So
I'm somewhat inclined to just fix this in master.

pg_plan_advice also seems to be affected and is missing from the
original patch, a generic fix like this handles that too and all
future cases.