pgsql: meson: Add initial version of meson based build system

Started by Andres Freundover 3 years ago5 messageshackers

andres@anarazel.de

over 3 years ago

meson: Add initial version of meson based build system

Autoconf is showing its age, fewer and fewer contributors know how to wrangle
it. Recursive make has a lot of hard to resolve dependency issues and slow
incremental rebuilds. Our home-grown MSVC build system is hard to maintain for
developers not using Windows and runs tests serially. While these and other
issues could individually be addressed with incremental improvements, together
they seem best addressed by moving to a more modern build system.

After evaluating different build system choices, we chose to use meson, to a
good degree based on the adoption by other open source projects.

We decided that it's more realistic to commit a relatively early version of
the new build system and mature it in tree.

This commit adds an initial version of a meson based build system. It supports
building postgres on at least AIX, FreeBSD, Linux, macOS, NetBSD, OpenBSD,
Solaris and Windows (however only gcc is supported on aix, solaris). For
Windows/MSVC postgres can now be built with ninja (faster, particularly for
incremental builds) and msbuild (supporting the visual studio GUI, but
building slower).

Several aspects (e.g. Windows rc file generation, PGXS compatibility, LLVM
bitcode generation, documentation adjustments) are done in subsequent commits
requiring further review. Other aspects (e.g. not installing test-only
extensions) are not yet addressed.

When building on Windows with msbuild, builds are slower when using a visual
studio version older than 2019, because those versions do not support
MultiToolTask, required by meson for intra-target parallelism.

The plan is to remove the MSVC specific build system in src/tools/msvc soon
after reaching feature parity. However, we're not planning to remove the
autoconf/make build system in the near future. Likely we're going to keep at
least the parts required for PGXS to keep working around until all supported
versions build with meson.

Some initial help for postgres developers is at
https://wiki.postgresql.org/wiki/Meson

With contributions from Thomas Munro, John Naylor, Stone Tickle and others.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Author: Peter Eisentraut <peter@eisentraut.org>
Reviewed-By: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: /messages/by-id/20211012083721.hvixq4pnh2pixr3j@alap3.anarazel.de

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be

Christoph Berg

myon@debian.org

almost 2 years ago

In reply to: Andres Freund (#1)

Re: pgsql: meson: Add initial version of meson based build system

Re: Andres Freund

https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be

This commit broke VPATH builds when the original source directory
contains symlinks.

The $PWD is /home/myon/postgresql/pg/master, but the actual directory
is /home/myon/projects/postgresql/pg/postgresql. When I
mkdir build; cd build && ../configure
there, I get a build directory missing a lot of files/directories:

$ ls build/
config.log config.status* GNUmakefile meson.build src/
$ ls build/src/
backend/ include/ interfaces/ Makefile.global Makefile.port@
$ ls build/src/backend/
port/

Given there are no other changes I think this bit is at fault:

Modified Files
--------------
configure.ac | 6 +

+# Ensure that any meson build directories would reconfigure and see that
+# there's a conflicting in-tree build and can error out.
+if test "$vpath_build"="no"; then
+  touch meson.build
+fi

Christoph

Andres Freund

andres@anarazel.de

almost 2 years ago

In reply to: Christoph Berg (#2)

Re: pgsql: meson: Add initial version of meson based build system

Hi,

Uh, huh.

On 2024-04-17 15:42:28 +0200, Christoph Berg wrote:

Re: Andres Freund

https://git.postgresql.org/pg/commitdiff/e6927270cd18d535b77cbe79c55c6584351524be

This commit broke VPATH builds when the original source directory
contains symlinks.

I.e. a symlink to the source directory, not a symlink inside the source
directory.

Given there are no other changes I think this bit is at fault:

Modified Files
--------------
configure.ac | 6 +
+# Ensure that any meson build directories would reconfigure and see that
+# there's a conflicting in-tree build and can error out.
+if test "$vpath_build"="no"; then
+  touch meson.build
+fi

Argh, this is missing spaces around the '=', leading to the branch always
being entered.

What I don't understand is how that possibly could affect the prep_buildtree
step, that happens earlier.

Hm.

Uh, I don't think it does? Afaict this failure is entirely unrelated to 'touch
meson.build'? From what I can tell the problem is that config/prep_buildtree
is invoked with the symlinked path, and that that doesn't seem to work:

bash -x /home/andres/src/postgresql-via-symlink/config/prep_buildtree /home/andres/src/postgresql-via-symlink .
++ basename /home/andres/src/postgresql-via-symlink/config/prep_buildtree
+ me=prep_buildtree
+ help='Usage: prep_buildtree sourcetree [buildtree]'
+ test -z /home/andres/src/postgresql-via-symlink
+ test x/home/andres/src/postgresql-via-symlink = x--help
+ unset CDPATH
++ cd /home/andres/src/postgresql-via-symlink
++ pwd
+ sourcetree=/home/andres/src/postgresql-via-symlink
++ cd .
++ pwd
+ buildtree=/tmp/pgs
++ find /home/andres/src/postgresql-via-symlink -type d '(' '(' -name CVS -prune ')' -o '(' -name .git -prune ')' -o -print ')'
++ grep -v '/home/andres/src/postgresql-via-symlink/doc/src/sgml/\+'
++ find /home/andres/src/postgresql-via-symlink -name Makefile -print -o -name GNUmakefile -print
++ grep -v /home/andres/src/postgresql-via-symlink/doc/src/sgml/images/
+ exit 0

Note that the find does not return anything.

Greetings,

Andres Freund

Christoph Berg

myon@debian.org

almost 2 years ago

In reply to: Andres Freund (#3)

Re: pgsql: meson: Add initial version of meson based build system

Re: Andres Freund

This commit broke VPATH builds when the original source directory
contains symlinks.

I.e. a symlink to the source directory, not a symlink inside the source
directory.

Yes.

Argh, this is missing spaces around the '=', leading to the branch always
being entered.

Glad I found at least something :)

Uh, I don't think it does? Afaict this failure is entirely unrelated to 'touch
meson.build'? From what I can tell the problem is that config/prep_buildtree
is invoked with the symlinked path, and that that doesn't seem to work:

Apparently I messed up both the git bisect run and manually
confirm the problem later. Trying again now, the problem has been
existing at least since 2002, probably earlier.

I've been using this directory layout for years, apparently so far
I've always only used non-VPATH builds or dpkg-buildpackage, which
probably canonicalizes the path before building, given it works.

Since no one else has been complaining, it might not be worth fixing.

Sorry for the noise!

Christoph

Andres Freund

andres@anarazel.de

almost 2 years ago

In reply to: Christoph Berg (#4)

Re: pgsql: meson: Add initial version of meson based build system

Hi,

On 2024-04-18 10:54:18 +0200, Christoph Berg wrote:

Re: Andres Freund

This commit broke VPATH builds when the original source directory
contains symlinks.

Argh, this is missing spaces around the '=', leading to the branch always
being entered.

Glad I found at least something :)

Yep :). I pushed a fix to that now.

I've been using this directory layout for years, apparently so far
I've always only used non-VPATH builds or dpkg-buildpackage, which
probably canonicalizes the path before building, given it works.

I wonder if perhaps find's behaviour might have changed at some point?

Since no one else has been complaining, it might not be worth fixing.

I'm personally not excited about working on fixing this, but if somebody else
wants to spend the cycles to make this work reliably...

It's certainly interesting that we have some code worrying about symlinks in
configure.ac:
# prepare build tree if outside source tree
# Note 1: test -ef might not exist, but it's more reliable than `pwd`.
# Note 2: /bin/pwd might be better than shell's built-in at getting
# a symlink-free name.

But we only use this to determine if we're doing a vpath build, not as the
path passed to prep_buildtree...

Greetings,

Andres Freund