BUG #15889: PostgreSQL failed to build due to error MSB8020 with MSVC on windows

On Wed, Jul 3, 2019 at 11:11 AM PG Bug reporting form
<noreply@postgresql.org> wrote:

Environment:
Windows Server 2016 + VS2017 + PostgreSQL master branch latest srouce
code.

In that enviroment, what version of nmake are you using?

nmake /?

Regards,

Juan José Santamaría Flecha

(Adding Hari who has helped with this work.)

On Wed, Jul 03, 2019 at 09:10:34AM +0000, PG Bug reporting form wrote:

PostgreSQL failed to build due to error MSB8020: The build tools for v142
(Platform Toolset = 'v142') cannot be found with MSVC on windows. It can be
first reproduced on master branch 2b1394fc2b52a2573d08aa626e7b49568f27464e
reversion which is 'Add support for Visual Studio 2019 in build scripts'.

Further investigation we found that VS2017 does not seem to be supported in
this commit. We mainly use open source projects to test VS compilers.
Currently we are using VS2017 and there is no plan to migrate to VS2019.
Could you please help look at this issue and provide a work around to us to
solve this issue? Thanks a lot!

Mumble. VS 2017 should be supported in our builds.

It happens that we do have a buildfarm member called bowerbird which
uses Visual Studio 2017, and this is not complaining at all:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&amp;dt=2019-07-02%2018%3A30%3A56

So at first sight the logic we have does not seem that wrong because
it has already tested the commit which adds VS 2019 :)

Failures:
C:\Program Files (x86)\Microsoft Visual
Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(57,5):
error MSB8020: The build tools for v142 (Platform Toolset = 'v142') cannot
be found. To build using the v142 build tools, please install v142 build
tools. Alternatively, you may upgrade to the current Visual Studio tools by
selecting the Project menu or right-click the solution, and then selecting
"Retarget solution". [D:\PostgreSQL\src\postgres.vcxproj]

Looking at the code, it looks to be an effect of the version checking
in CreateProject()/MSBuildProject.pm, causing a VC2019Project object
to be generated instead of a VC2017Project one. Your scripts seem to
be rather advanced, and I have little idea how your installation looks
like. What's the return result of DetermineVisualStudioVersion()?
With VS 2017, the result is assumed to be between 14.10 and 14.20 as
far as I know. You can also know that by looking at the version of
nmake shipped with your installation using the native tool terminal
for example. I guess that in your case the version of nmake is
between 14.20.XX.YY and 15.00.NN.PP?
--
Michael

On Wed, Jul 3, 2019 at 1:36 PM Michael Paquier <michael@paquier.xyz> wrote:

Looking at the code, it looks to be an effect of the version checking
in CreateProject()/MSBuildProject.pm, causing a VC2019Project object
to be generated instead of a VC2017Project one. Your scripts seem to
be rather advanced, and I have little idea how your installation looks
like. What's the return result of DetermineVisualStudioVersion()?
With VS 2017, the result is assumed to be between 14.10 and 14.20 as
far as I know. You can also know that by looking at the version of
nmake shipped with your installation using the native tool terminal
for example. I guess that in your case the version of nmake is
between 14.20.XX.YY and 15.00.NN.PP?

In that building environment, if you edit
".\src\tools\msvc\buildenv.pl" to add the following line:

$ENV{MSBFLAGS}="/p:PlatformToolset=v141";

Does it work as workaround for you?

Anyhow, could you please share what nmake version are you using?

Regards,

Juan José Santamaría Flecha

On Thu, Jul 04, 2019 at 10:18:44AM +0200, Juan José Santamaría Flecha wrote:

In that building environment, if you edit
".\src\tools\msvc\buildenv.pl" to add the following line:

$ENV{MSBFLAGS}="/p:PlatformToolset=v141";

Does it work as workaround for you?

Anyhow, could you please share what nmake version are you using?

As a side thought, we could also move away from nmake when it comes to
find out the version of Visual Studio used if Microsoft is lazy with
nmake's version number. For example we could consider using
"MSBuild.exe -version" and parse its output. This reports 16.1.XX.YY
for me with VS 2019, and 14.0.XX.YY with VS 2015.

Not sure how portable that it though past those versions. Still, on
HEAD and v12, we only support VS 2013 and newer versions, so we could
integrate with it easily.
--
Michael

On Thu, Jul 4, 2019 at 10:32 AM Michael Paquier <michael@paquier.xyz> wrote:

Not sure how portable that it though past those versions. Still, on
HEAD and v12, we only support VS 2013 and newer versions, so we could
integrate with it easily.

Not sure if that is actually an improvement, I would like to hear some
feedback from the reporter.

Regards,

Juan José Santamaría Flecha

On Thu, Jul 04, 2019 at 08:12:30PM +0200, Juan José Santamaría Flecha wrote:

Not sure if that is actually an improvement, I would like to hear some
feedback from the reporter.

Well that could move us from a position where we rely on a non-linear
and messed up version history to something which is purely linear,
based on a command which actually refers to the core of Visual Studio
build scripts. So that seems worth the benefit.

Let's see what the reporter says on the matter.
--
Michael

On Wed, 3 Jul 2019 at 21:35, Michael Paquier <michael@paquier.xyz> wrote:

(Adding Hari who has helped with this work.)

On Wed, Jul 03, 2019 at 09:10:34AM +0000, PG Bug reporting form wrote:

PostgreSQL failed to build due to error MSB8020: The build tools for v142
(Platform Toolset = 'v142') cannot be found with MSVC on windows. It

can be

first reproduced on master branch

2b1394fc2b52a2573d08aa626e7b49568f27464e

reversion which is 'Add support for Visual Studio 2019 in build

scripts'.

Further investigation we found that VS2017 does not seem to be supported

in

this commit. We mainly use open source projects to test VS compilers.
Currently we are using VS2017 and there is no plan to migrate to VS2019.
Could you please help look at this issue and provide a work around to us

to

solve this issue? Thanks a lot!

Mumble. VS 2017 should be supported in our builds.

It happens that we do have a buildfarm member called bowerbird which
uses Visual Studio 2017, and this is not complaining at all:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=bowerbird&amp;dt=2019-07-02%2018%3A30%3A56

So at first sight the logic we have does not seem that wrong because
it has already tested the commit which adds VS 2019 :)

During the development of this patch, I tested various combinations with
both VS 2017 and
VS 2019 and didn't find any problems.

Failures:
C:\Program Files (x86)\Microsoft Visual

Studio\2017\Enterprise\Common7\IDE\VC\VCTargets\Microsoft.Cpp.Platform.targets(57,5):

error MSB8020: The build tools for v142 (Platform Toolset = 'v142')

cannot

be found. To build using the v142 build tools, please install v142 build
tools. Alternatively, you may upgrade to the current Visual Studio

tools by

selecting the Project menu or right-click the solution, and then

selecting

"Retarget solution". [D:\PostgreSQL\src\postgres.vcxproj]

Looking at the code, it looks to be an effect of the version checking
in CreateProject()/MSBuildProject.pm, causing a VC2019Project object
to be generated instead of a VC2017Project one. Your scripts seem to
be rather advanced, and I have little idea how your installation looks
like. What's the return result of DetermineVisualStudioVersion()?
With VS 2017, the result is assumed to be between 14.10 and 14.20 as
far as I know. You can also know that by looking at the version of
nmake shipped with your installation using the native tool terminal
for example. I guess that in your case the version of nmake is
between 14.20.XX.YY and 15.00.NN.PP?

I feel the reason for failure may be the project files are generated with
VS 2019
first and later try to use the same project files with VS 2017?

May be reporter can clarify it.

Regards,
Haribabu Kommi

On Thu, 4 Jul 2019 at 18:32, Michael Paquier <michael@paquier.xyz> wrote:

On Thu, Jul 04, 2019 at 10:18:44AM +0200, Juan José Santamaría Flecha
wrote:

In that building environment, if you edit
".\src\tools\msvc\buildenv.pl" to add the following line:

$ENV{MSBFLAGS}="/p:PlatformToolset=v141";

Does it work as workaround for you?

Anyhow, could you please share what nmake version are you using?

As a side thought, we could also move away from nmake when it comes to
find out the version of Visual Studio used if Microsoft is lazy with
nmake's version number. For example we could consider using
"MSBuild.exe -version" and parse its output. This reports 16.1.XX.YY
for me with VS 2019, and 14.0.XX.YY with VS 2015.

Not sure how portable that it though past those versions. Still, on
HEAD and v12, we only support VS 2013 and newer versions, so we could
integrate with it easily.

I agree that we can think of using MSBuild.exe instead of nmake. I am also
not sure whether MSBuild is available in all the visual studio versions
[1]: https://en.wikipedia.org/wiki/MSBuild

[1]: https://en.wikipedia.org/wiki/MSBuild

Regards,
Haribabu Kommi

On Wed, Jul 10, 2019 at 07:53:51PM +1000, Haribabu Kommi wrote:

I agree that we can think of using MSBuild.exe instead of nmake. I am also
not sure whether MSBuild is available in all the visual studio versions
[1].

[1] - https://en.wikipedia.org/wiki/MSBuild

Well, we could consider switching to MSBuild.exe only on HEAD and v12,
where we don't need to support the same range of VS versions. That
sounds like a rather safe bet, and I'd be rather fine to integrate
that if it makes the build scripts more solid.
--
Michael

On Wed, Jul 10, 2019 at 2:51 PM Michael Paquier <michael@paquier.xyz> wrote:

On Wed, Jul 10, 2019 at 07:53:51PM +1000, Haribabu Kommi wrote:

I agree that we can think of using MSBuild.exe instead of nmake. I am also
not sure whether MSBuild is available in all the visual studio versions
[1].

[1] - https://en.wikipedia.org/wiki/MSBuild

Well, we could consider switching to MSBuild.exe only on HEAD and v12,
where we don't need to support the same range of VS versions. That
sounds like a rather safe bet, and I'd be rather fine to integrate
that if it makes the build scripts more solid.

Instead of substituting an executable for another, maybe using an
environment variable from the vcvarsall script will make it more
straightforward.

Regards,

Juan José Santamaría Flecha