windows config.pl question

Started by Dmitry Markmanover 5 years ago8 messages
#1Dmitry Markman
dmarkman@mac.com

Hi

I’d like to add icu/openssl support to my postgresql build on windows

documentation says that I have to modify config.pl file, however it's not clear what exactly I have to do

config-default.pl for example has the following line

icu => undef, # --with-icu=<path>

so, if I want to add icu support what exactly should I do?

should I replace undef with path?

icu => <path_to_icu_install_area>,

is it correct?

if it’s correct does build support UNC paths?

thanks in advance

Dimitry Markman
Dmitry Markman

#2Michael Paquier
michael@paquier.xyz
In reply to: Dmitry Markman (#1)
Re: windows config.pl question

On Thu, Jul 30, 2020 at 06:55:28AM -0400, Dmitry Markman wrote:

icu => <path_to_icu_install_area>,

is it correct?

Exactly.

if it’s correct does build support UNC paths?

Yes, these work. One take to be aware of is that the quoting of the
paths can be annoying. FWIW, I just use single quotes with normal
slashes as that's a no-brainer, say:
openssl => 'C:/OpenSSL-hoge/',

If you are able to break the scripts with a given path, this would
likely be a bug we should address. For example, we had lately
complains about the build scripts breaking once you inserted spaces in
the python or OpenSSL paths (see beb2516 and ad7595b).
--
Michael

#3Dmitry Markman
dmarkman@mac.com
In reply to: Michael Paquier (#2)
Re: windows config.pl question

Hi Michael, thanks a lot

I figured it out, UNC works indeed

however I found at least 2 problems (at least in our 3p harness)

1. in our configuration openssl executable went to lib (I don’t know why), so I had to change line in Configure script
and point to openssl.exe file. Sure I probably can change our makefile to create bin directory and put openssl.exe there
but it’s not my file, maybe later

2. if PostgreSQL is situated on network drive that mapped to say disk z:, then build script failed:

Z:\3p\derived\win64\PostgreSQL\source\src\tools\msvc>build
Detected hardware platform: Win32
Generating win32ver.rc for src/backend
Generating win32ver.rc for src/timezone
Generating win32ver.rc for src/backend/snowball
Generating win32ver.rc for src/pl/plpgsql/src
Generating win32ver.rc for src/backend/replication/libpqwalreceiver
Generating win32ver.rc for src/backend/replication/pgoutput
Generating win32ver.rc for src/interfaces/ecpg/pgtypeslib

. . . . . . . . . . . .

Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
Build started 7/30/2020 5:52:12 PM.
Project "Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" on node 1 (default targets).
Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln.metaproj : error MSB4126: The specified solution configuration "Release
|x64" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g. M
SBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the defaul
t solution configuration. [Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln]
Done Building Project "Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" (default targets) -- FAILED.

Build FAILED.

"Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" (default target) (1) ->
(ValidateSolutionConfiguration target) ->
Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln.metaproj : error MSB4126: The specified solution configuration "Relea
se|x64" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g.
MSBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the defa
ult solution configuration. [Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln]

0 Warning(s)
1 Error(s)

Time Elapsed 00:00:00.37

Z:\3p\derived\win64\PostgreSQL\source\src\tools\msvc>

the same works just fine if it’s on c: drive

all PostgreSQL distribution is in the Z:\3p\derived\win64\PostgreSQL\source folder

network UNC path is mapped to Z:

thanks again for your help

dm

Show quoted text

On Jul 30, 2020, at 9:59 PM, Michael Paquier <michael@paquier.xyz> wrote:

On Thu, Jul 30, 2020 at 06:55:28AM -0400, Dmitry Markman wrote:

icu => <path_to_icu_install_area>,

is it correct?

Exactly.

if it’s correct does build support UNC paths?

Yes, these work. One take to be aware of is that the quoting of the
paths can be annoying. FWIW, I just use single quotes with normal
slashes as that's a no-brainer, say:
openssl => 'C:/OpenSSL-hoge/',

If you are able to break the scripts with a given path, this would
likely be a bug we should address. For example, we had lately
complains about the build scripts breaking once you inserted spaces in
the python or OpenSSL paths (see beb2516 and ad7595b).
--
Michael

#4Dmitry Markman
dmarkman@mac.com
In reply to: Dmitry Markman (#3)
Re: windows config.pl question

sorry I meant file src/tools/msvc/Solution.pm

routine sub GetOpenSSLVersion

has the follwing line:

qq("$self->{options}->{openssl}\\bin\\openssl.exe" version 2>&1);

in our distribution openssl.exe isn’t in the $self->{options}->{openssl}\bin\ location

dm

Show quoted text

On Jul 30, 2020, at 10:25 PM, Dmitry Markman <dmarkman@mac.com> wrote:

Hi Michael, thanks a lot

I figured it out, UNC works indeed

however I found at least 2 problems (at least in our 3p harness)

1. in our configuration openssl executable went to lib (I don’t know why), so I had to change line in Configure script
and point to openssl.exe file. Sure I probably can change our makefile to create bin directory and put openssl.exe there
but it’s not my file, maybe later

2. if PostgreSQL is situated on network drive that mapped to say disk z:, then build script failed:

Z:\3p\derived\win64\PostgreSQL\source\src\tools\msvc>build
Detected hardware platform: Win32
Generating win32ver.rc for src/backend
Generating win32ver.rc for src/timezone
Generating win32ver.rc for src/backend/snowball
Generating win32ver.rc for src/pl/plpgsql/src
Generating win32ver.rc for src/backend/replication/libpqwalreceiver
Generating win32ver.rc for src/backend/replication/pgoutput
Generating win32ver.rc for src/interfaces/ecpg/pgtypeslib

. . . . . . . . . . . .

Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
Build started 7/30/2020 5:52:12 PM.
Project "Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" on node 1 (default targets).
Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln.metaproj : error MSB4126: The specified solution configuration "Release
|x64" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g. M
SBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the defaul
t solution configuration. [Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln]
Done Building Project "Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" (default targets) -- FAILED.

Build FAILED.

"Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln" (default target) (1) ->
(ValidateSolutionConfiguration target) ->
Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln.metaproj : error MSB4126: The specified solution configuration "Relea
se|x64" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g.
MSBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the defa
ult solution configuration. [Z:\3p\derived\win64\PostgreSQL\source\pgsql.sln]

0 Warning(s)
1 Error(s)

Time Elapsed 00:00:00.37

Z:\3p\derived\win64\PostgreSQL\source\src\tools\msvc>

the same works just fine if it’s on c: drive

all PostgreSQL distribution is in the Z:\3p\derived\win64\PostgreSQL\source folder

network UNC path is mapped to Z:

thanks again for your help

dm

On Jul 30, 2020, at 9:59 PM, Michael Paquier <michael@paquier.xyz> wrote:

On Thu, Jul 30, 2020 at 06:55:28AM -0400, Dmitry Markman wrote:

icu => <path_to_icu_install_area>,

is it correct?

Exactly.

if it’s correct does build support UNC paths?

Yes, these work. One take to be aware of is that the quoting of the
paths can be annoying. FWIW, I just use single quotes with normal
slashes as that's a no-brainer, say:
openssl => 'C:/OpenSSL-hoge/',

If you are able to break the scripts with a given path, this would
likely be a bug we should address. For example, we had lately
complains about the build scripts breaking once you inserted spaces in
the python or OpenSSL paths (see beb2516 and ad7595b).
--
Michael

#5Michael Paquier
michael@paquier.xyz
In reply to: Dmitry Markman (#4)
Re: windows config.pl question

On Thu, Jul 30, 2020 at 11:16:01PM -0400, Dmitry Markman wrote:

sorry I meant file src/tools/msvc/Solution.pm

routine sub GetOpenSSLVersion

has the follwing line:

qq("$self->{options}->{openssl}\\bin\\openssl.exe" version 2>&1);

in our distribution openssl.exe isn’t in the $self->{options}->{openssl}\bin\ location

No idea what you are using as OpenSSL installation , so I cannot say
for sure. FWIW, the scripts in the code tree are made compatible with
what we suggest in the documentation here:
https://www.postgresql.org/docs/current/install-windows-full.html#id-1.6.4.8.8
--
Michael

#6Dmitry Markman
dmarkman@mac.com
In reply to: Michael Paquier (#5)
Re: windows config.pl question

Hi Michael, I found the problem

command

cl /?

gives different answer if you start that command from c: or from z: (where z: is mapped drive)

if current directory is on c:

then cl /? returns

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional>cl /?
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27042 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

C/C++ COMPILER OPTIONS

-OPTIMIZATION-

/O1 maximum optimizations (favor space) /O2 maximum optimizations (favor speed)
/Ob<n> inline expansion (default n=0) /Od disable optimizations (default)
/Og enable global optimization /Oi[-] enable intrinsic functions
/Os favor code space /Ot favor code speed
/Ox optimizations (favor speed)
/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:
blend - a combination of optimizations for several different x64 processors
AMD64 - 64-bit AMD processors
INTEL64 - Intel(R)64 architecture processors
ATOM - Intel(R) Atom(TM) processors

-CODE GENERATION-

/Gu[-] ensure distinct functions have distinct addresses
/Gw[-] separate global variables for linker
/GF enable read-only string pooling /Gm[-] enable minimal rebuild
/Gy[-] separate functions for linker /GS[-] enable security checks
/GR[-] enable C++ RTTI /GX[-] enable C++ EH (same as /EHsc)

. . . . . . . . . . . . . . .

but if I issue that command if the current folder is on z:

Z:\>cl /?
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27042 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

from other hand

cl -help

returns consisten answer from c: or from z:

so platform wasn’t identified properly if build started from z:

after I changed cl /? to cl -help

build and install went successfully

all but one test (tablespace) finished successfully.

that one failure also related to network drive, because

after build finished I copied whole directory that contain PostgreSQL distro to

c: and run tests everything went smoothly

thanks

dm

Show quoted text

On Jul 31, 2020, at 5:43 AM, Michael Paquier <michael@paquier.xyz> wrote:

On Thu, Jul 30, 2020 at 11:16:01PM -0400, Dmitry Markman wrote:

sorry I meant file src/tools/msvc/Solution.pm

routine sub GetOpenSSLVersion

has the follwing line:

qq("$self->{options}->{openssl}\\bin\\openssl.exe" version 2>&1);

in our distribution openssl.exe isn’t in the $self->{options}->{openssl}\bin\ location

No idea what you are using as OpenSSL installation , so I cannot say
for sure. FWIW, the scripts in the code tree are made compatible with
what we suggest in the documentation here:
https://www.postgresql.org/docs/current/install-windows-full.html#id-1.6.4.8.8
--
Michael

#7Michael Paquier
michael@paquier.xyz
In reply to: Dmitry Markman (#6)
Re: windows config.pl question

On Fri, Jul 31, 2020 at 10:41:46PM -0400, Dmitry Markman wrote:

but if I issue that command if the current folder is on z:

Z:\>cl /?
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27042 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

from other hand

Interesting. We rely on the presence of "favor:" in the output to
determine which platform to use, aka x64 or Win32.

cl -help

returns consistent answer from c: or from z:

so platform wasn’t identified properly if build started from z:

What's the output of cl -help on "z:" in this case? Is the exact same
output as "cl /?" or "cl -help" on c: generated? I have to admit that
I don't really know why things would behave this way, but Windows is a
platform full of undiscovered mysteries, and I have never seen the
output of cl being an issue even for some of my company work, which
uses stuff much more fancy than the normal way of compiling on
Windows, requiring me to patch a bit the scripts of src/tools/msvc/ in
a different way.
--
Michael

#8Dmitry Markman
dmarkman@mac.com
In reply to: Michael Paquier (#7)
Re: windows config.pl question

cl -help works the same

on z: or on c:

and it’s equivalent to the output of cl /? from c:

Z:\>cat cl_out.txt | grep favor
/O1 maximum optimizations (favor space) /O2 maximum optimizations (favor speed)
/Os favor code space /Ot favor code speed
/Ox optimizations (favor speed)
/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:

where cl_out.txt is result cl -help > cl_out.txt

Show quoted text

On Jul 31, 2020, at 10:58 PM, Michael Paquier <michael@paquier.xyz> wrote:

On Fri, Jul 31, 2020 at 10:41:46PM -0400, Dmitry Markman wrote:

but if I issue that command if the current folder is on z:

Z:\>cl /?
Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27042 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

usage: cl [ option... ] filename... [ /link linkoption... ]

from other hand

Interesting. We rely on the presence of "favor:" in the output to
determine which platform to use, aka x64 or Win32.

cl -help

returns consistent answer from c: or from z:

so platform wasn’t identified properly if build started from z:

What's the output of cl -help on "z:" in this case? Is the exact same
output as "cl /?" or "cl -help" on c: generated? I have to admit that
I don't really know why things would behave this way, but Windows is a
platform full of undiscovered mysteries, and I have never seen the
output of cl being an issue even for some of my company work, which
uses stuff much more fancy than the normal way of compiling on
Windows, requiring me to patch a bit the scripts of src/tools/msvc/ in
a different way.
--
Michael