meson: Add support for building with precompiled headers
Hi,
This is a patch split off from the initial meson thread [1]/messages/by-id/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de as it's
functionally largely independent (as suggested in [2]/messages/by-id/e0c44fb2-8b66-a4b9-b274-7ed3a1a0ab74@enterprisedb.com).
Using precompiled headers substantially speeds up building for windows, due to
the vast amount of headers included via windows.h. A cross build from
linux targetting mingw goes from
994.11user 136.43system 0:31.58elapsed 3579%CPU
to
422.41user 89.05system 0:14.35elapsed 3562%CPU
The wins on windows are similar-ish (but I don't have a system at hand just
now for actual numbers). Targetting other operating systems the wins are far
smaller (tested linux, macOS, FreeBSD).
This is particularly interesting for cfbot, which spends a lot of time
building on windows. It also makes developing on windows less painful as the
gains are bigger when compiling incrementally, because the precompiled headers
don't typically have to be rebuilt.
As a prerequisite this requires changing the way FD_SETSIZE is defined when
targetting windows.
When using precompiled headers we cannot override macros in system headers
from within .c files, as headers are already processed before the #define in
the C file is reached.
A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.
We likely should move away from using select() in those places, but that's a
larger change.
Michael, CCing you wrt the second patch, as Thomas noticed [3]/messages/by-id/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com that you were
looking at where to define FD_SETSIZE.
Greetings,
Andres Freund
[1]: /messages/by-id/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de
[2]: /messages/by-id/e0c44fb2-8b66-a4b9-b274-7ed3a1a0ab74@enterprisedb.com
[3]: /messages/by-id/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
[4]: /messages/by-id/20190826054000.GE7005@paquier.xyz
Andres Freund <andres@anarazel.de> writes:
When using precompiled headers we cannot override macros in system headers
from within .c files, as headers are already processed before the #define in
the C file is reached.
A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.
Color me confused, but how does it work to #define that from the command
line if it can't be overridden from within the program?
regards, tom lane
Hi,
On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
When using precompiled headers we cannot override macros in system headers
from within .c files, as headers are already processed before the #define in
the C file is reached.A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.Color me confused, but how does it work to #define that from the command
line if it can't be overridden from within the program?
If specified on the commandline it's also used when generating the precompiled
header - of course that's not possible when it's just #define'd in some .c
file.
Greetings,
Andres Freund
Andres Freund <andres@anarazel.de> writes:
On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
Color me confused, but how does it work to #define that from the command
line if it can't be overridden from within the program?
If specified on the commandline it's also used when generating the precompiled
header - of course that's not possible when it's just #define'd in some .c
file.
Ah, so there's a separate cache of precompiled headers for each set of
compiler command-line arguments? Got it.
regards, tom lane
Hi,
On 2022-10-05 16:21:55 -0400, Tom Lane wrote:
Andres Freund <andres@anarazel.de> writes:
On 2022-10-05 16:09:14 -0400, Tom Lane wrote:
Color me confused, but how does it work to #define that from the command
line if it can't be overridden from within the program?If specified on the commandline it's also used when generating the precompiled
header - of course that's not possible when it's just #define'd in some .c
file.Ah, so there's a separate cache of precompiled headers for each set of
compiler command-line arguments? Got it.
Worse, it builds the precompiled header for each "target" (static/shared lib,
executable), right now. Hence I've only added them for targets that have
multiple .c files. I've been planning to submit an improvement to meson that
does what you propose, it'd not be hard, but before it's actually usable, it
didn't seem worth investing time in that.
Greetings,
Andres Freund
On 05.10.22 21:08, Andres Freund wrote:
This is a patch split off from the initial meson thread [1] as it's
functionally largely independent (as suggested in [2]).Using precompiled headers substantially speeds up building for windows, due to
the vast amount of headers included via windows.h. A cross build from
linux targetting mingw goes from
These patches look ok to me. I can't really comment on the Windows
details, but it sounds all reasonable.
Small issue:
+override CFLAGS += -DFD_SETSIZE=1024
(and similar)
should be CPPFLAGS.
Hi,
On 2022-10-06 09:06:42 +0200, Peter Eisentraut wrote:
On 05.10.22 21:08, Andres Freund wrote:
This is a patch split off from the initial meson thread [1] as it's
functionally largely independent (as suggested in [2]).Using precompiled headers substantially speeds up building for windows, due to
the vast amount of headers included via windows.h. A cross build from
linux targetting mingw goes fromThese patches look ok to me. I can't really comment on the Windows details,
but it sounds all reasonable.
Thanks for reviewing!
Small issue:
+override CFLAGS += -DFD_SETSIZE=1024
(and similar)
should be CPPFLAGS.
Pushed with that adjusted.
Greetings,
Andres Freund