Windows meson build

On 05.11.24 07:32, Kohei Harikae (Fujitsu) wrote:

I do not use pkgconf in my Windows environment.
In my Windows environment, I could not build the following OSS with meson.
- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd

[1]thread, I created a patch like the one in the attached file, and now I can build.
Would you like to be able to build OSS with Windows meson without using pkgconf?

The problem I see with these patches is that they create a separate code
path for Windows. Right now, all platforms use the same code path for
discovering the dependencies, which is nice for maintainability.

So I suggest the solution here is for you to install pkgconf and use it.

Hi,

On 2024-11-05 06:32:51 +0000, Kohei Harikae (Fujitsu) wrote:

I do not use pkgconf in my Windows environment.
In my Windows environment, I could not build the following OSS with meson.
- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd

[1]thread, I created a patch like the one in the attached file, and now I can build.
Would you like to be able to build OSS with Windows meson without using pkgconf?

You can use pkgconf or cmake for the dependencies. I don't want to add "raw"
dependency handling for every dependency, they each build in too many variants
for that to be a sensible investment of time.

Greetings,

Andres Freund

Hi,

I apologize for this late reply.

As you advised, I was able to create the ".pc" file manually and set the dependencies using pkgconf.
Thank you.

However, I have read the PostgreSQL Documentation and did not realize that pkgconf is required.

17.4.3.2. PostgreSQL Features
-Dgssapi
On many systems, the GSSAPI system (a part of the MIT Kerberos installation) is not installed in a location that is searched by default (e.g., /usr/include, /usr/lib).
In those cases, PostgreSQL will query pkg-config to detect the required compiler and linker options.
-Dlibxml
To use a libxml2 installation that is in an unusual location, you can set pkg-config-related environment variables (see its documentation).

Only a few options have a pkg-config description.
It would be easier to understand pkg-config in the "17.4.3. meson setup Options" section.
Below is my proposal for the document description.
How do you think about this?

17.4.3. meson setup Options
PostgreSQL will query pkg-config to detect the required compiler and linker options.
To use module installed in an unusual location for each feature (e.g. -Dgssapi, -Dlibxml), set pkg-config-related environment variables (see its documentation).

Regards,
Kohei Harikae

-----Original Message-----
From: Andres Freund <andres@anarazel.de>
Sent: Thursday, November 7, 2024 4:05 AM
To: Harikae, Kohei/張替 浩平 <harikae.kohei@fujitsu.com>
Cc: 'pgsql-hackers@lists.postgresql.org' <pgsql-hackers@lists.postgresql.org>
Subject: Re: Windows meson build

Hi,

On 2024-11-05 06:32:51 +0000, Kohei Harikae (Fujitsu) wrote:

I do not use pkgconf in my Windows environment.
In my Windows environment, I could not build the following OSS with meson.
- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd

[1]thread, I created a patch like the one in the attached file, and now I can build.
Would you like to be able to build OSS with Windows meson without using pkgconf?

You can use pkgconf or cmake for the dependencies. I don't want to add "raw"
dependency handling for every dependency, they each build in too many variants for that to be a sensible investment of time.

Greetings,

Andres Freund

Kohei Harikae (Fujitsu) писал(а) 2024-12-18 05:05:
Hi!

Hi,

I apologize for this late reply.

As you advised, I was able to create the ".pc" file manually and set
the dependencies using pkgconf.
Thank you.

However, I have read the PostgreSQL Documentation and did not realize
that pkgconf is required.

17.4.3.2. PostgreSQL Features
-Dgssapi
On many systems, the GSSAPI system (a part of the MIT Kerberos
installation) is not installed in a location that is searched by
default (e.g., /usr/include, /usr/lib).
In those cases, PostgreSQL will query pkg-config to detect the
required compiler and linker options.
-Dlibxml
To use a libxml2 installation that is in an unusual location, you can
set pkg-config-related environment variables (see its documentation).

Only a few options have a pkg-config description.
It would be easier to understand pkg-config in the "17.4.3. meson setup
Options" section.
Below is my proposal for the document description.
How do you think about this?

17.4.3. meson setup Options
PostgreSQL will query pkg-config to detect the required compiler and
linker options.
To use module installed in an unusual location for each feature (e.g.
-Dgssapi, -Dlibxml), set pkg-config-related environment variables (see
its documentation).

Meson has more straighforward solution. It uses environment variable
PKG_CONFIG_PATH, that can store the list of directories with all .pc
files. Meson uses this list and in this specific order to build project.
As wrote in meson documentation (in the source of open source project):
# Instead of relying on pkg-config or pkgconf to provide -L flags in a
# specific order, we reorder library paths ourselves, according to th
# order specified in PKG_CONFIG_PATH. See:
# https://github.com/mesonbuild/meson/issues/4271
#
It would be more clear information in PostgreSQL documentation, that all
directories with pc files are listed in one variable.

By the way, if you need to add , f.e. libxml library to postgreSQL
build, you can
1) build and install it using , f.e. vcpkg:
vcpkg install libxml2:windows-x64 --x-install-root=c:\pbuild\libxml2
2) add path to pc directory to PKG_CONFIG_PATH
SET
PKG_CONFIG_PATH=c:\pbuild\libxml2\winsows-x64\lib\pkgconfig;%PKG_CONFIG_PATH%
3) make steps with build
meson setup c:\builddir\... --prefix=c:\postgresdir\...

It is just one install and one manipulation with environment variable.
No indeterminate behaviour with other variables used by specific
packages and a libraries order.

Regards,
Kohei Harikae

-----Original Message-----
From: Andres Freund <andres@anarazel.de>
Sent: Thursday, November 7, 2024 4:05 AM
To: Harikae, Kohei/張替 浩平 <harikae.kohei@fujitsu.com>
Cc: 'pgsql-hackers@lists.postgresql.org'
<pgsql-hackers@lists.postgresql.org>
Subject: Re: Windows meson build

Hi,

On 2024-11-05 06:32:51 +0000, Kohei Harikae (Fujitsu) wrote:

I do not use pkgconf in my Windows environment.
In my Windows environment, I could not build the following OSS with
meson.
- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd

[1]thread, I created a patch like the one in the attached file, and
now I can build.
Would you like to be able to build OSS with Windows meson without
using pkgconf?

You can use pkgconf or cmake for the dependencies. I don't want to add
"raw"
dependency handling for every dependency, they each build in too many
variants for that to be a sensible investment of time.

Greetings,

Andres Freund

--
Best regards,

Vladlen Popolitov.

Hi,

Thank you for your advice.
I added a sample setting PKG_CONFIG_PATH based on your advice.

How do you think about this?

Regards,
Kohei Harikae

-----Original Message-----
From: Vladlen Popolitov <v.popolitov@postgrespro.ru>
Sent: Wednesday, December 25, 2024 7:28 PM
To: Harikae, Kohei/張替 浩平 <harikae.kohei@fujitsu.com>
Cc: 'Andres Freund' <andres@anarazel.de>; 'pgsql-hackers@lists.postgresql.org' <pgsql-hackers@lists.postgresql.org>
Subject: Re: Windows meson build

Kohei Harikae (Fujitsu) писал(а) 2024-12-18 05:05:
Hi!

Hi,

I apologize for this late reply.

As you advised, I was able to create the ".pc" file manually and set
the dependencies using pkgconf.
Thank you.

However, I have read the PostgreSQL Documentation and did not realize
that pkgconf is required.

17.4.3.2. PostgreSQL Features
-Dgssapi
On many systems, the GSSAPI system (a part of the MIT Kerberos
installation) is not installed in a location that is searched by
default (e.g., /usr/include, /usr/lib).
In those cases, PostgreSQL will query pkg-config to detect the
required compiler and linker options.
-Dlibxml
To use a libxml2 installation that is in an unusual location, you
can set pkg-config-related environment variables (see its documentation).

Only a few options have a pkg-config description.
It would be easier to understand pkg-config in the "17.4.3. meson
setup Options" section.
Below is my proposal for the document description.
How do you think about this?

17.4.3. meson setup Options
PostgreSQL will query pkg-config to detect the required compiler and
linker options.
To use module installed in an unusual location for each feature (e.g.
-Dgssapi, -Dlibxml), set pkg-config-related environment variables (see
its documentation).

Meson has more straighforward solution. It uses environment variable PKG_CONFIG_PATH, that can store the list of directories with all .pc files. Meson uses this list and in this specific order to build project.
As wrote in meson documentation (in the source of open source project):
# Instead of relying on pkg-config or pkgconf to provide -L flags in a # specific order, we reorder library paths ourselves, according to th # order specified in PKG_CONFIG_PATH. See:
# https://github.com/mesonbuild/meson/issues/4271
#
It would be more clear information in PostgreSQL documentation, that all directories with pc files are listed in one variable.

By the way, if you need to add , f.e. libxml library to postgreSQL build, you can
1) build and install it using , f.e. vcpkg:
vcpkg install libxml2:windows-x64 --x-install-root=c:\pbuild\libxml2
2) add path to pc directory to PKG_CONFIG_PATH
SET
PKG_CONFIG_PATH=c:\pbuild\libxml2\winsows-x64\lib\pkgconfig;%PKG_CONFIG_PATH%
3) make steps with build
meson setup c:\builddir\... --prefix=c:\postgresdir\...

It is just one install and one manipulation with environment variable.
No indeterminate behaviour with other variables used by specific packages and a libraries order.

Regards,
Kohei Harikae

-----Original Message-----
From: Andres Freund <andres@anarazel.de>
Sent: Thursday, November 7, 2024 4:05 AM
To: Harikae, Kohei/張替 浩平 <harikae.kohei@fujitsu.com>
Cc: 'pgsql-hackers@lists.postgresql.org'
<pgsql-hackers@lists.postgresql.org>
Subject: Re: Windows meson build

Hi,

On 2024-11-05 06:32:51 +0000, Kohei Harikae (Fujitsu) wrote:

I do not use pkgconf in my Windows environment.
In my Windows environment, I could not build the following OSS with
meson.
- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd

[1]thread, I created a patch like the one in the attached file, and
now I can build.
Would you like to be able to build OSS with Windows meson without
using pkgconf?

You can use pkgconf or cmake for the dependencies. I don't want to
add "raw"
dependency handling for every dependency, they each build in too many
variants for that to be a sensible investment of time.

Greetings,

Andres Freund

--
Best regards,

Vladlen Popolitov.

Kohei Harikae (Fujitsu) писал(а) 2024-12-27 04:31:

Hi,

Thank you for your advice.
I added a sample setting PKG_CONFIG_PATH based on your advice.

How do you think about this?

Regards,
Kohei Harikae

Hi,
From my point of view it looks good. Actual information is added,
excessive information is removed.
I think in the phrase:
"add the directory where the .pc file resides to PKG_CONFIG_PATH."
better to add wording, that PKG_CONFIG_PATH is the list of directories
separated
by PATH separator ( ; in Windows, : in POSIX). Otherwise the reader has
to
search additional information about this variable in meson
documentation.

--
Best regards,

Vladlen Popolitov.

Vladlen Popolitov писал(а) 2024-12-27 09:57:

Kohei Harikae (Fujitsu) писал(а) 2024-12-27 04:31:

Hi,

Thank you for your advice.
I added a sample setting PKG_CONFIG_PATH based on your advice.

How do you think about this?

Regards,
Kohei Harikae

Hi,
From my point of view it looks good. Actual information is added,
excessive information is removed.
I think in the phrase:
"add the directory where the .pc file resides to PKG_CONFIG_PATH."
better to add wording, that PKG_CONFIG_PATH is the list of directories
separated
by PATH separator ( ; in Windows, : in POSIX). Otherwise the reader has
to
search additional information about this variable in meson
documentation.

Hi

I think, it is better to add the other options how to configure external
libraries, if they do not have .pc file:
add meson options defined by PostgreSQL for its configuration (gettext
library example)

-Dextra_include_dirs=c:\postgres\gettext\x64-windows\include,c:\otherlibs\include
-Dextra_lib_dirs=c:\postgres\gettext\x64-windows\lib,c:\otherlibs\lib

extra_include_dirs and extra_lib_dirs are options defined by PostgreSQL.
This options are
comma separated lists of directories, if every value does not contain
comma itself.

--
Best regards,

Vladlen Popolitov.

Hi,

I think in the phrase:
"add the directory where the .pc file resides to PKG_CONFIG_PATH."
better to add wording, that PKG_CONFIG_PATH is the list of directories
separated by PATH separator ( ; in Windows, : in POSIX). Otherwise the
reader has to search additional information about this variable in
meson documentation.

Thank you for your advice.
I added the description of PATH separator based on your advice.

I think, it is better to add the other options how to configure external
libraries, if they do not have .pc file:
add meson options defined by PostgreSQL for its configuration (gettext
library example)

-Dextra_include_dirs=c:\postgres\gettext\x64-windows\include,c:\otherlibs\include
-Dextra_lib_dirs=c:\postgres\gettext\x64-windows\lib,c:\otherlibs\lib

extra_include_dirs and extra_lib_dirs are options defined by PostgreSQL.
This options are
comma separated lists of directories, if every value does not contain
comma itself.

- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd
These libraries could not be built by setting extra_include_dirs and extra_lib_dirs.
These libraries may not have .pc file created.
For example, if I use msc (Visual Studio) to build zLib, zlib.pc is not created.
Could you tell me which libraries can use this option?

Regards,
Kohei Harikae

Kohei Harikae (Fujitsu) писал(а) 2025-01-06 16:57:

Hi,

- 0001 icu
- 0002 libxml
- 0003 libxslt
- 0004 lz4
- 0005 tcl
- 0006 zlib
- 0007 zstd
These libraries could not be built by setting extra_include_dirs and
extra_lib_dirs.
These libraries may not have .pc file created.
For example, if I use msc (Visual Studio) to build zLib, zlib.pc is not
created.
Could you tell me which libraries can use this option?

Regards,
Kohei Harikae

Hi!

I don't have my Windows PC near me just now, I take information from my
memory regarding these
libraries.

I use Windows package manager vcpkg to build these libraries, and vcpkg
uses msvc (Visual Studio).
As I remember, icu, libxml2, libxslt build by vcpkg have .pc file.
Really pc file is easy to
create by hands the text file with
a) product name and version
b) relative (from .pc file location) path to libs
c) relative path to includes
Libraries with .pc file can be listed in environment variable
PKG_CONFIG_PATH.

gettext does not have pc file, and lib and include location must be
included in extra_include_dirs
and extra_lib_dirs options, that are PostgreSQL options in meson (not
meson itself options) added
for libraries without .pc file.

Steps to build PostgreSQL under Windsows using meson and vcpkg
1) create build.cmd for all commands
2) install vcpkg to any location (except network drive, it has problems
with it. Hard drive C: is OK)
3) install all libraries:
vcpkg install libxml2
etc...
4) look for all libraries lib/pkgconf library and find .pc file. All pc
file paths add to
SET PKG_CONFIG_PATH=... in build.cmd
5) all libraries without pc file - add to -Dextra_include_dirs= and
-Dextra_lib_dirs for meson command,
that prepares the build directory - add it to build.cmd
6) add in the beginning build.cmd this or correct path for your system
call "C:\Program Files\Microsoft Visual
Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" x64
7) run build.cmd

This library configuration should work. I recommend install libraries to
special location for every
build version to control exact location and library version. vcpkg has
options for library version
and install directory.

--
Best regards,

Vladlen Popolitov.

Hi,

Thank you for your advice.
Based on your advice, I have added a description of the vcpkg and -Dextra_include_dirs options.
How do you think about this?

Regards,
Kohei Harikae

Kohei Harikae (Fujitsu) писал(а) 2025-01-10 16:43:

Hi,

Thank you for your advice.
Based on your advice, I have added a description of the vcpkg and
-Dextra_include_dirs options.
How do you think about this?

Regards,
Kohei Harikae

Hi,

Looks good! If you create commitfest entry, I would make review.

--
Best regards,

Vladlen Popolitov.

Hi,

My previous patch had whitespace issues.
I recreated the patch.
I will create a commitfest with this patch.

Regards,
Kohei Harikae

The following review has been posted through the commitfest application:
make installcheck-world: tested, failed
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation: tested, failed

Hi!

This patch clarify the options used by meson to build PostgreSQL, how
to included libraries and include files from dependencies:
1) environment variable PKG_CONFIG_PATH
2) options -Dextra_include_dirs= and -Dextra_lib_dirs
This information useful for Windows builds and any other system,
where meson build is used.

+1 for commit

The new status of this patch is: Ready for Committer

Vladlen Popolitov писал(а) 2025-02-13 17:58:

The following review has been posted through the commitfest
application:
make installcheck-world: tested, failed
Implements feature: tested, failed
Spec compliant: tested, failed
Documentation: tested, failed

Hi!

Previous email has text "failed". It is generated by "send review"
function,
I do not know, how to change it (tried in other browsers).

Nothing failed in this patch.

--
Best regards,

Vladlen Popolitov.

On 13.02.25 12:38, Vladlen Popolitov wrote:

Vladlen Popolitov писал(а) 2025-02-13 17:58:

The following review has been posted through the commitfest application:
make installcheck-world:  tested, failed
Implements feature:       tested, failed
Spec compliant:           tested, failed
Documentation:            tested, failed

Hi!

 Previous email has text "failed". It is generated by "send review"
function,
I do not know, how to change it (tried in other browsers).

Nothing failed in this patch.

This patch is pretty old but it was still hanging around as open in the
commitfest app.

I realize that the way the PostgreSQL build process finds all the
dependencies is pretty complicated right now. With configure some of
them are found the old way using CPPFLAGS/LDFLAGS, some via specific
foo-config scripts, some via pkg-config, some via specific custom code.
With meson, similarly, but the actual details are different in each
case. Maybe we should document all of this accurately, or maybe it
would be unmaintainable. In any case, I don't think your patch has
enough coverage to make this better, it just describes some specific
scenario that works for you, and it moves some text out of context where
it doesn't make sense anymore. Maybe this kind of information is better
in a how-to document or blog post, and from there we could generalize
the information and maybe feed it back into the documentation.