BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form <
noreply@postgresql.org> wrote:

in src\tools\msvc\Solution.pm the following test is not sufficient
$self->{platform} =
($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
Microsoft (R) C/C++ Optimizing Compiler Version
19.28.29336 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

I see no bug here, that expression is looking for the line:
"/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one
of:", which is one of the first printed options.
To make sure this is working in your environment, you should see "Detected
hardware platform: x64" when building.

Regards,

Juan José Santamaría Flecha

Hi,

Yes, I see

Don’t know why when I’m doing the cl /? In the Visual Studio path of the x64 Native Command Prompt, the response contains all the help (including the /flavor: ….)

Once I change to the postgresql\src\tools\msvc for to build, the ‘cl /?’ Only gives back 2 lines and say invalid option.

But I also remarked that if you pass ‘cl -?’, it works correctly in both case. Do you think it is possible to change the ‘cl /?’ by ‘cl -?’ ?

Thanks for your help

Regards

Robert

From: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com>
Sent: Saturday, January 16, 2021 10:16 PM
To: robionekenobi@bluewin.ch; PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form < <mailto:noreply@postgresql.org> noreply@postgresql.org> wrote:

in src\tools\msvc\Solution.pm the following test is not sufficient
$self->{platform} =
($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

I see no bug here, that expression is looking for the line: "/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:", which is one of the first printed options.

To make sure this is working in your environment, you should see "Detected hardware platform: x64" when building.

Regards,

Juan José Santamaría Flecha

Hi again,

I searched a little bit more, and this occurs only if the drive where I run the ‘cl /?’ Is a ‘Subst Drive’

Therefor, I’ve changed this to a real Drive, and the error does no more occurs. It is a bug in windows with ‘Subst drive’ probably

So, you can either close the bug, or change the ‘cl /?’ by a ‘cl /help’ (which is also an official cl option)

Regards

Robert

From: robionekenobi@bluewin.ch <robionekenobi@bluewin.ch>
Sent: Sunday, January 17, 2021 9:31 AM
To: 'Juan José Santamaría Flecha' <juanjo.santamaria@gmail.com>; 'PostgreSQL mailing lists' <pgsql-bugs@lists.postgresql.org>
Subject: RE: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

Hi,

Yes, I see

Don’t know why when I’m doing the cl /? In the Visual Studio path of the x64 Native Command Prompt, the response contains all the help (including the /flavor: ….)

Once I change to the postgresql\src\tools\msvc for to build, the ‘cl /?’ Only gives back 2 lines and say invalid option.

But I also remarked that if you pass ‘cl -?’, it works correctly in both case. Do you think it is possible to change the ‘cl /?’ by ‘cl -?’ ?

Thanks for your help

Regards

Robert

From: Juan José Santamaría Flecha <juanjo.santamaria@gmail.com <mailto:juanjo.santamaria@gmail.com> >
Sent: Saturday, January 16, 2021 10:16 PM
To: robionekenobi@bluewin.ch <mailto:robionekenobi@bluewin.ch> ; PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org <mailto:pgsql-bugs@lists.postgresql.org> >
Subject: Re: BUG #16825: When building on Windows, cl /? retrun 'x64' not AMD64 and the build does not create x64 environment

On Fri, Jan 15, 2021 at 9:21 AM PG Bug reporting form < <mailto:noreply@postgresql.org> noreply@postgresql.org> wrote:

in src\tools\msvc\Solution.pm the following test is not sufficient
$self->{platform} =
($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 'Win32';
because the cl /? returns:
Microsoft (R) C/C++ Optimizing Compiler Version 19.28.29336 for x64
Copyright (C) Microsoft Corporation. All rights reserved.

I see no bug here, that expression is looking for the line: "/favor:<blend|AMD64|INTEL64|ATOM> select processor to optimize for, one of:", which is one of the first printed options.

To make sure this is working in your environment, you should see "Detected hardware platform: x64" when building.

Regards,

Juan José Santamaría Flecha

On Sun, Jan 17, 2021 at 7:22 PM <robionekenobi@bluewin.ch> wrote:

Hi again,

I searched a little bit more, and this occurs only if the drive where I
run the ‘cl /?’ Is a ‘Subst Drive’

Therefor, I’ve changed this to a real Drive, and the error does no more
occurs. It is a bug in windows with ‘Subst drive’ probably

So, you can either close the bug, or change the ‘cl /?’ by a ‘cl /help’
(which is also an official cl option)

Thanks for the reproducible example, it behaves like that for me too.

I guess 'cl' is interpreting "/?" as filename, not as option, when the
current path is a ‘Subst Drive’. If you call 'cl /? foo' it works, and also
if you create a one-letter named dummy file inside the ‘Subst Drive’.

I would close this report noting that using a ‘Subst Drive’ is problematic
anyhow, as other scripts might also fail when navigating the expected
folder layout.

Regards,

Juan José Santamaría Flecha

On Mon, Jan 18, 2021 at 11:42:33PM +0100, Juan José Santamaría Flecha wrote:

On Sun, Jan 17, 2021 at 7:22 PM <robionekenobi@bluewin.ch> wrote:
I guess 'cl' is interpreting "/?" as filename, not as option, when the
current path is a ‘Subst Drive’. If you call 'cl /? foo' it works, and also
if you create a one-letter named dummy file inside the ‘Subst Drive’.

I would close this report noting that using a ‘Subst Drive’ is problematic
anyhow, as other scripts might also fail when navigating the expected
folder layout.

If you patch the scripts by changing the option used by the cl
command, are there cases where the installation able to complete when
using a subst driver? It seems to me that if things work could work
even in this case and result in a correct installation, then there is
an argument for making the MSVC scripts easier to use for the
end-user.
--
Michael

On Tue, Jan 19, 2021 at 6:23 AM Michael Paquier <michael@paquier.xyz> wrote:

If you patch the scripts by changing the option used by the cl
command, are there cases where the installation able to complete when
using a subst driver? It seems to me that if things work could work
even in this case and result in a correct installation, then there is
an argument for making the MSVC scripts easier to use for the
end-user.

The scripts assume that the 'msvc' folder is three levels deep,
'src/tools/msvc'. If you respect that, you can build with the 'cl /help'
patch. Please find attached the patch for so.

Regards,

Juan José Santamaría

On Tue, Jan 19, 2021 at 09:31:22AM +0100, Juan José Santamaría Flecha wrote:

The scripts assume that the 'msvc' folder is three levels deep,
'src/tools/msvc'. If you respect that, you can build with the 'cl /help'
patch. Please find attached the patch for so.

So, I have been playing with subst and some virtual drives, and this
allows to run builds if for example pointing to the top of a git repo,
which is fancy.

Requiring at least three repository levels is something that we are
going to need anyway as a build requires a full access to the tree,
but I really see no reason to help users with such configurations as
that's basically free for us to do so. I can see the difference of
output between cl /? and /help for virtual drives, and that's really
surprising, but I see no reason to not apply this patch as this breaks
nothing and /help is an official option.

If there are no objections, I am planning to apply this patch to
HEAD.
--
Michael

On Wed, Jan 20, 2021 at 04:24:47PM +0900, Michael Paquier wrote:

If there are no objections, I am planning to apply this patch to
HEAD.

Done as of 733d670. There may be a point in back-patching and
classifying that as a bug, but I have played it conservative as that's
an improvement and nobody really complained about that AFAIK.
--
Michael

Hi.

On 21.1.2021. 3:12, Michael Paquier wrote:

On Wed, Jan 20, 2021 at 04:24:47PM +0900, Michael Paquier wrote:

If there are no objections, I am planning to apply this patch to
HEAD.

Done as of 733d670. There may be a point in back-patching and
classifying that as a bug, but I have played it conservative as that's
an improvement and nobody really complained about that AFAIK.
--
Michael

Thanks for patching this.

And actually I did report it a while ago, just hadn't found the time
to follow up and rebase the patch to develop:

------

Hi.

I'm not sure what the process is here so just attempting a polite ping.

I've posted a message (reattached to the bottom of this e-mail) about
this problem, including a fix for it, almost a month about and there has
thus far been no response nor can I see the change applied to

https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/tools/msvc/Solution.pm;hb=HEAD

Please let me know if I should be posting about this somewhere else.

Kind regards,
Jurko Gospodnetić

On 7.10.2020. 18:08, PG Bug reporting form wrote:

The following bug has been logged on the website:

Bug reference: 16660
Logged by: Jurko Gospodnetić
Email address: jurko.gospodnetic@pke.hr
PostgreSQL version: 9.6.19
Operating system: Windows
Description:

If you run the 64-bit postgres build from a virtual subst drive (see
Windows `help subst` command), it will fail.

Internally it will fail to detect that the used Visual C++ compiler
executable `cl.exe` has been configured for producing 64-bit targets
and will thus configure the generated Visual Studio projects &
solution for building 32-bit targets even through the used compiler
will actually be producing 64-bit ones.

Even deeper, the original problem stems from `cl.exe` behaviour, which
seems to produce no compiler option output when invoked as `cl /?`
from a subst drive. On one occasion it was reported that it did show
compiler options on some subst drives but not others, but this could
not be reproduced later on so such behaviour can be at most considered
random.

Invoking the executable as `cl /help` works correctly on all tested
drives. Go figure.

Here's a patch fixing the problem based on the code in the 9.6.19
distribution:

From a3ccfe68dbe5190f743cd13a4fc90faaf6478370 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jurko=20Gospodneti=C4=87?= <jurko.gospodnetic@pke.hr>
Date: Wed, 7 Oct 2020 16:17:29 +0200
Subject: [PATCH] patch compiler architecture detection on virtual subst
drives

---
src/tools/msvc/Solution.pm | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 7ec7120..29ebab5 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -75,8 +75,13 @@ sub DeterminePlatform
   {
      my $self = shift;
-    # Examine CL help output to determine if we are in 32 or 64-bit mode.
-    my $output = `cl /? 2>&1`;
+    # examine CL help output to determine if we are in 32 or 64-bit mode
+    #
+    # we intentionally use `cl /help` as opposed to `cl /?` as the 
former works
+    # when the current drive is a virtual `subst` drive (see `help 
subst`) while
+    # the latter does not (noticed at least with compiler executable 
version
+    # `Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27043 for 
x64`)
+    my $output = `cl /help 2>&1`;
      $? >> 8 == 0 or die "cl command not found";
      $self->{platform} = ($output =~ /^\/favor:<.+AMD64/m) ? 'x64' : 
'Win32';
      print "Detected hardware platform: $self->{platform}\n";

On Fri, Jan 22, 2021 at 01:11:30PM +0100, Jurko Gospodnetić wrote:

And actually I did report it a while ago, just hadn't found the time to
follow up and rebase the patch to develop:

Indeed, my apologies for that. pgsql-bugs is a list with a lot of
traffic, so it is easy to miss something. I can see that this thread
is not registered in the commit fest app, making this even easier to
forget:
https://commitfest.postgresql.org/
--
Michael