Postgres Windows build system doesn't work with python installed in Program Files

On Thu, Apr 30, 2020 at 03:06:08PM +0300, Victor Wagner wrote:

Fix is very simple, see attach.

Patch is made against REL_12_STABLE, but probably applicable to other
versions as well.

Indeed, thanks.

my $pythonprog = "import sys;print(sys.prefix);"
. "print(str(sys.version_info[0])+str(sys.version_info[1]))";
my $prefixcmd =
-		  $solution->{options}->{python} . "\\python -c \"$pythonprog\"";
+		'"' . $solution->{options}->{python} . "\\python\" -c \"$pythonprog\"";
my $pyout = `$prefixcmd`;
die "Could not query for python version!\n" if $?;
my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);

This reminds me of ad7595b. Wouldn't it be better to use qq() here?
--
Michael

В Fri, 1 May 2020 17:52:15 +0900
Michael Paquier <michael@paquier.xyz> пишет:

On Thu, Apr 30, 2020 at 03:06:08PM +0300, Victor Wagner wrote:

Fix is very simple, see attach.

Patch is made against REL_12_STABLE, but probably applicable to
other versions as well.

Indeed, thanks.

my $pythonprog = "import sys;print(sys.prefix);"
.
"print(str(sys.version_info[0])+str(sys.version_info[1]))"; my
$prefixcmd =
-		  $solution->{options}->{python} . "\\python -c
\"$pythonprog\"";
+		'"' . $solution->{options}->{python} . "\\python\"
-c \"$pythonprog\""; my $pyout = `$prefixcmd`;
die "Could not query for python version!\n" if $?;
my ($pyprefix, $pyver) = split(/\r?\n/, $pyout);  

This reminds me of ad7595b. Wouldn't it be better to use qq() here?

Maybe. But probably original author of this code was afraid of using
too long chain of ->{} in the string substitution.

So, I left this style n place.

Nonetheless, using qq wouldn't save us from doubling backslashes.

--

Em qui., 30 de abr. de 2020 às 09:06, Victor Wagner <vitus@wagner.pp.ru>
escreveu:

Collegues,

Accidently I've come over minor bug in the Mkvcbuild.pm.
It happens, that it doesn't tolerate spaces in the $config->{python}
path, because it want to call python in order to find out version,
prefix and so on, and doesn't properly quote command.

Fix is very simple, see attach.

Patch is made against REL_12_STABLE, but probably applicable to other
versions as well.

I don't know if it applies to the same case, but from the moment I
installed python on the development machine, the Postgres build stopped
working correctly.
Although perl, flex and bison are available in the path, the build does not
generate files that depend on flex and bison.

Running bison on src/backend/bootstrap/bootparse.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/bootstrap/bootscanner.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running bison on src/backend/parser/gram.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
etc

Warning from build.pl
Use of uninitialized value $ARGV[0] in uc at build.pl line 44.
Use of uninitialized value $ARGV[0] in uc at build.pl line 48.

regards,
Ranier Vilela

On Sun, May 03, 2020 at 04:23:24PM -0300, Ranier Vilela wrote:

I don't know if it applies to the same case, but from the moment I
installed python on the development machine, the Postgres build stopped
working correctly.
Although perl, flex and bison are available in the path, the build does not
generate files that depend on flex and bison.

Are you following the instructions of the documentation? Here is a
link to them:
https://www.postgresql.org/docs/devel/install-windows-full.html

My guess is that you would be just missing a PATH configuration or
such because python enforced a new setting?

Warning from build.pl
Use of uninitialized value $ARGV[0] in uc at build.pl line 44.
Use of uninitialized value $ARGV[0] in uc at build.pl line 48.

Hmm. We have buildfarm machines using the MSVC scripts and Python,
see for example woodloose. And note that @ARGV would be normally
defined, so your warning looks fishy to me.
--
Michael

On Mon, May 4, 2020 at 7:58 AM Michael Paquier <michael@paquier.xyz> wrote:

Warning from build.pl
Use of uninitialized value $ARGV[0] in uc at build.pl line 44.
Use of uninitialized value $ARGV[0] in uc at build.pl line 48.

Hmm. We have buildfarm machines using the MSVC scripts and Python,
see for example woodloose. And note that @ARGV would be normally
defined, so your warning looks fishy to me.

I think these are two different issues, python PATH and build.pl warnings.
For the later, you can check woodloose logs and see the warning after
commit 8f00d84afc.

Regards,

Juan José Santamaría Flecha

Em seg., 4 de mai. de 2020 às 02:58, Michael Paquier <michael@paquier.xyz>
escreveu:

On Sun, May 03, 2020 at 04:23:24PM -0300, Ranier Vilela wrote:

I don't know if it applies to the same case, but from the moment I
installed python on the development machine, the Postgres build stopped
working correctly.
Although perl, flex and bison are available in the path, the build does

not

generate files that depend on flex and bison.

Are you following the instructions of the documentation? Here is a
link to them:
https://www.postgresql.org/docs/devel/install-windows-full.html

My environment was ok and worked 100%, compiling with msvc 2019 (64 bits).

My guess is that you would be just missing a PATH configuration or
such because python enforced a new setting?

Perl and flex and bison, are in the path, no doubt.

Warning from build.pl
Use of uninitialized value $ARGV[0] in uc at build.pl line 44.
Use of uninitialized value $ARGV[0] in uc at build.pl line 48.

Hmm. We have buildfarm machines using the MSVC scripts and Python,
see for example woodloose. And note that @ARGV would be normally
defined, so your warning looks fishy to me.

I'll redo from the beginning.
1. Make empty directory postgres
2. Clone postgres
3. Call msvc 2019 (64 bits) env batch
4. Setup path to perl, bison and flex
set path=%path%;c:\perl;\bin;c:\bin

5. C:\dll>perl -V
Summary of my perl5 (revision 5 version 30 subversion 1) configuration:

Platform:
osname=MSWin32
osvers=10.0.18363.476
archname=MSWin32-x64-multi-thread
uname='Win32 strawberry-perl 5.30.1.1 #1 Fri Nov 22 02:24:29 2019 x64'

6. C:\dll>bison -V
bison (GNU Bison) 2.7
Written by Robert Corbett and Richard Stallman.

7. C:\dll>flex -V
flex 2.6.4

8. cd\dll\postgres\src\tools\msvc
9. build

results:
...
PrepareForBuild:
Creating directory ".\Release\postgres\".
Creating directory ".\Release\postgres\postgres.tlog\".
InitializeBuildStatus:
Creating ".\Release\postgres\postgres.tlog\unsuccessfulbuild" because
"AlwaysCreate" was specified.
CustomBuild:
Running bison on src/backend/bootstrap/bootparse.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/bootstrap/bootscanner.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running bison on src/backend/parser/gram.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/parser/scan.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running bison on src/backend/replication/repl_gram.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/replication/repl_scanner.l
Running bison on src/backend/replication/syncrep_gram.y
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/replication/syncrep_scanner.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running bison on src/backend/utils/adt/jsonpath_gram.y
Running flex on src/backend/utils/adt/jsonpath_scan.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
Running flex on src/backend/utils/misc/guc-file.l
'perl' nao é reconhecido como um comando interno
ou externo, um programa operável ou um arquivo em lotes.
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5):
error MSB6006: "cmd.exe" exited with code 9009.
[C:\dll\postgres\postgres.vcxproj]
Done Building Project "C:\dll\postgres\postgres.vcxproj" (default targets)
-- FAILED.

Done Building Project "C:\dll\postgres\pgsql.sln" (default targets) --
FAILED.

...

Build FAILED.

"C:\dll\postgres\pgsql.sln" (default target) (1) ->
"C:\dll\postgres\cyrillic_and_mic.vcxproj" (default target) (2) ->
"C:\dll\postgres\postgres.vcxproj" (default target) (3) ->
(CustomBuild target) ->
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5):
error MSB6006: "cmd.exe" exited with code 900
9. [C:\dll\postgres\postgres.vcxproj]

"C:\dll\postgres\pgsql.sln" (default target) (1) ->
"C:\dll\postgres\initdb.vcxproj" (default target) (32) ->
"C:\dll\postgres\libpgfeutils.vcxproj" (default target) (33) ->
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5):
error MSB6006: "cmd.exe" exited with code 900
9. [C:\dll\postgres\libpgfeutils.vcxproj]

"C:\dll\postgres\pgsql.sln" (default target) (1) ->
"C:\dll\postgres\ecpg.vcxproj" (default target) (124) ->
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5):
error MSB6006: "cmd.exe" exited with code 900
9. [C:\dll\postgres\ecpg.vcxproj]

"C:\dll\postgres\pgsql.sln" (default target) (1) ->
"C:\dll\postgres\isolationtester.vcxproj" (default target) (129) ->
C:\Program Files (x86)\Microsoft Visual
Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(231,5):
error MSB6006: "cmd.exe" exited with code 900
9. [C:\dll\postgres\isolationtester.vcxproj]

0 Warning(s)
4 Error(s)

No warnings, this time.

best regards,
Ranier Vilela

On Mon, May 04, 2020 at 09:45:54AM +0200, Juan José Santamaría Flecha wrote:

I think these are two different issues, python PATH and build.pl warnings.
For the later, you can check woodloose logs and see the warning after
commit 8f00d84afc.

Oh, indeed. I somewhat managed to miss these in the logs of the
buildfarm. What if we refactored the code of build.pl so as we'd
check first if $ARGV[0] is defined or not? If not defined, then we
need to have a release-quality build for all the components. How does
that sound? Something not documented is that using "release" as first
argument enforces also a release-quality build for all the components,
so we had better not break that part.
--
Michael

On Mon, May 4, 2020 at 2:18 PM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, May 04, 2020 at 09:45:54AM +0200, Juan José Santamaría Flecha
wrote:

I think these are two different issues, python PATH and build.pl

warnings.

For the later, you can check woodloose logs and see the warning after
commit 8f00d84afc.

Oh, indeed. I somewhat managed to miss these in the logs of the
buildfarm. What if we refactored the code of build.pl so as we'd
check first if $ARGV[0] is defined or not? If not defined, then we
need to have a release-quality build for all the components. How does
that sound? Something not documented is that using "release" as first
argument enforces also a release-quality build for all the components,
so we had better not break that part.

+1, seems like the way to go to me.

Regards,

Juan José Santamaría Flecha

On Fri, May 01, 2020 at 12:48:17PM +0300, Victor Wagner wrote:

Maybe. But probably original author of this code was afraid of using
too long chain of ->{} in the string substitution.

So, I left this style n place.

Nonetheless, using qq wouldn't save us from doubling backslashes.

Looking at this part in more details, I find the attached much more
readable. I have been able to test it on my own Windows environment
and the problem gets fixed (I have reproduced the original problem as
well).
--
Michael

В Tue, 5 May 2020 15:45:48 +0900
Michael Paquier <michael@paquier.xyz> пишет:

On Fri, May 01, 2020 at 12:48:17PM +0300, Victor Wagner wrote:

Maybe. But probably original author of this code was afraid of using
too long chain of ->{} in the string substitution.

So, I left this style n place.

Nonetheless, using qq wouldn't save us from doubling backslashes.

Looking at this part in more details, I find the attached much more
readable. I have been able to test it on my own Windows environment

I agree, it is better.

and the problem gets fixed (I have reproduced the original problem as
well).

--

On Mon, May 04, 2020 at 03:42:20PM +0200, Juan José Santamaría Flecha wrote:

+1, seems like the way to go to me.

Attached is a patch for that and I have gone with a simple solution,
with some bonus comments about the way things happen. Here are the
patterns I tested for build.pl and the commands it generates, making
sure that we have the same commands with HEAD and the patch:
1) perl build.pl
msbuild pgsql.sln /verbosity:normal /p:Configuration=Release
2) perl build.pl debug
msbuild pgsql.sln /verbosity:normal /p:Configuration=Debug
3) perl build.pl release
msbuild pgsql.sln /verbosity:normal /p:Configuration=Release
4) perl build.pl foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Release
5) perl build.pl debug foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Debug
6) perl build.pl release foo
msbuild foo.vcxproj /verbosity:normal /p:Configuration=Release

The two warnings show up only in the first case, of course.
--
Michael

On Tue, May 5, 2020 at 4:06 PM Michael Paquier <michael@paquier.xyz> wrote:

On Mon, May 04, 2020 at 03:42:20PM +0200, Juan José Santamaría Flecha
wrote:

+1, seems like the way to go to me.

Attached is a patch for that and I have gone with a simple solution,
with some bonus comments about the way things happen.

This solves the issue.

Please forgive me if I am being too nitpicky, but I find the comments a
little too verbose, a usage format might be more visual and easier to
explain:

Usage: build [[CONFIGURATION] COMPONENT]

The options are case-insensitive.
CONFIGURATION sets the configuration to build, "debug" or "release" (by
default).
COMPONENT defines a component to build. An empty option means all
components.

Regards,

Juan José Santamaría Flecha

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..
--
Michael

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier <michael@paquier.xyz>
escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with path.It
works like this.
For some reason, which I haven't yet discovered, msbuild is ignoring the
path, where perl and bison and flex are.
Although it is being set, within the 64-bit compilation environment of msvc
2019.
I'm still investigating.

regards,
Ranier Vilela

Em qua., 6 de mai. de 2020 às 10:21, Ranier Vilela <ranier.vf@gmail.com>
escreveu:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier <michael@paquier.xyz>
escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with path.It
works like this.
For some reason, which I haven't yet discovered, msbuild is ignoring the
path, where perl and bison and flex are.
Although it is being set, within the 64-bit compilation environment of
msvc 2019.
I'm still investigating.

In fact perl, it is found, otherwise, neither build.pl would be working.
But within the perl environment, when the system call is made, in this
case, neither perl, bison, nor flex is found.

regards,
Ranier Vilela

Em qua., 6 de mai. de 2020 às 10:25, Ranier Vilela <ranier.vf@gmail.com>
escreveu:

Em qua., 6 de mai. de 2020 às 10:21, Ranier Vilela <ranier.vf@gmail.com>
escreveu:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier <michael@paquier.xyz>
escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with path.It
works like this.
For some reason, which I haven't yet discovered, msbuild is ignoring the
path, where perl and bison and flex are.
Although it is being set, within the 64-bit compilation environment of
msvc 2019.
I'm still investigating.

In fact perl, it is found, otherwise, neither build.pl would be working.
But within the perl environment, when the system call is made, in this
case, neither perl, bison, nor flex is found.

I'm using it like this, for now.

File pgbison.pl:
system("c:\\bin\\bison $headerflag $input -o $output");
File pgflex.pl:
system("c:\\bin\\flex $flexflags -o$output $input");
system("c:\\perl\\bin\\perl src\\tools\\fix-old-flex-code.pl
$output");

File Solution.pm:
system(
system('perl generate-lwlocknames.pl lwlocknames.txt');
system(
system(
system(
system(
system(
system(
system("perl create_help.pl ../../../doc/src/sgml/ref
sql_help");
system(
system(
system(
system(
system(
system('perl parse.pl < ../../../backend/parser/gram.y >
preproc.y');
system(

C:\dll\postgres\src\tools\msvc>\bin\grep bison *pm
File MSBuildProject.pm:
<Message
Condition="'\$(Configuration)|\$(Platform)'=='Debug|$self->{platform}'">Running
bison on $grammarFile</Message>
<Command
Condition="'\$(Configuration)|\$(Platform)'=='Debug|$self->{platform}'">c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgbison.pl" "$grammarFile"</Command>
<Message
Condition="'\$(Configuration)|\$(Platform)'=='Release|$self->{platform}'">Running
bison on $grammarFile</Message>
<Command
Condition="'\$(Configuration)|\$(Platform)'=='Release|$self->{platform}'">c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgbison.pl" "$grammarFile"</Command>

C:\dll\postgres\src\tools\msvc>\bin\grep flex *pm
File MSBuildProject.pm:
<Message
Condition="'\$(Configuration)|\$(Platform)'=='Debug|$self->{platform}'">Running
flex on $grammarFile</Message>
<Command
Condition="'\$(Configuration)|\$(Platform)'=='Debug|$self->{platform}'">c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgflex.pl" "$grammarFile"</Command>
<Message
Condition="'\$(Configuration)|\$(Platform)'=='Release|$self->{platform}'">Running
flex on $grammarFile</Message>
<Command
Condition="'\$(Configuration)|\$(Platform)'=='Release|$self->{platform}'">c:\\perl\\bin\\perl
"src\\tools\\msvc\\pgflex.pl" "$grammarFile"</Command>

regards,
Ranier Vilela

В Wed, 6 May 2020 10:21:41 -0300
Ranier Vilela <ranier.vf@gmail.com> пишет:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier
<michael@paquier.xyz> escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with
path.It works like this.

Perl has "magic" variable $^X which expands to full path to perl
executable, I wonder why build.pl doesn't use it to invoke secondary
perl scripts.

--

On 5/6/20 1:14 PM, Victor Wagner wrote:

В Wed, 6 May 2020 10:21:41 -0300
Ranier Vilela <ranier.vf@gmail.com> пишет:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier
<michael@paquier.xyz> escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with
path.It works like this.

Perl has "magic" variable $^X which expands to full path to perl
executable, I wonder why build.pl doesn't use it to invoke secondary
perl scripts.

We assume perl, flex and bison are in the PATH. That doesn't seem
unreasonable, it's worked well for quite a long time.

cheers

andrew

--
Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Em qua., 6 de mai. de 2020 às 14:14, Victor Wagner <vitus@wagner.pp.ru>
escreveu:

В Wed, 6 May 2020 10:21:41 -0300
Ranier Vilela <ranier.vf@gmail.com> пишет:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier
<michael@paquier.xyz> escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with
path.It works like this.

Perl has "magic" variable $^X which expands to full path to perl
executable, I wonder why build.pl doesn't use it to invoke secondary
perl scripts.

I still don't think it's necessary, it was working well.
My main suspicions are:
1. Path with spaces;
2. Incompatibility with < symbol, some suggest use &quot;

<Exec Command="&quot;

3. Msbuid.exe It has been updated (version 16.5.0)
4. Perl scripts increased the level of security.
5. My user do not have administrator rights.

regards,
Ranier Vilela

Em qua., 6 de mai. de 2020 às 15:19, Ranier Vilela <ranier.vf@gmail.com>
escreveu:

Em qua., 6 de mai. de 2020 às 14:14, Victor Wagner <vitus@wagner.pp.ru>
escreveu:

В Wed, 6 May 2020 10:21:41 -0300
Ranier Vilela <ranier.vf@gmail.com> пишет:

Em qua., 6 de mai. de 2020 às 09:53, Michael Paquier
<michael@paquier.xyz> escreveu:

On Tue, May 05, 2020 at 10:16:23AM +0300, Victor Wagner wrote:

I agree, it is better.

Thanks, applied and back-patched down to 9.5. Now for the second
problem of this thread..

Sorry, no clue yet.
I hacked the perl sources, to hardcoded perl, bison and flex with
path.It works like this.

Perl has "magic" variable $^X which expands to full path to perl
executable, I wonder why build.pl doesn't use it to invoke secondary
perl scripts.

I still don't think it's necessary, it was working well.
My main suspicions are:
1. Path with spaces;
2. Incompatibility with < symbol, some suggest use &quot;

<Exec Command="&quot;

3. Msbuid.exe It has been updated (version 16.5.0)
4. Perl scripts increased the level of security.
5. My user do not have administrator rights.

Cause found.

How it worked before
1. Call link from menu Visual Studio 2019: Auxiliary\Build\vcvars64.bat
That create a console with settings to compile on 64 bits.
2. Adjusting the path manually
set path=%path%;c:\perl\bin;c:\bin
3. Call build.bat

Hacking pgbison.pl, to print PATH, shows that the path inside pgbison.pl,
returned to being the original, without the addition of c:\perl\bin;c:\bin.
my $out = $ENV{PATH};
print "Path after system call=$out\n";
Path after system
call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
The final part lacks: c:\perl\bin;c:\bin

Now I need to find out why the path is being reset, within the perl scripts.

Cause: PATH being reset.

regards,
Ranier Vilela

On Wed, May 06, 2020 at 02:11:34PM -0400, Andrew Dunstan wrote:

We assume perl, flex and bison are in the PATH. That doesn't seem
unreasonable, it's worked well for quite a long time.

I recall that it is an assumption we rely on since MSVC scripts are
around, and that's rather easy to configure, so it seems to me that
changing things now would just introduce annoying changes for anybody
(developers, maintainers) using this stuff.
--
Michael

Em qua., 6 de mai. de 2020 às 21:08, Michael Paquier <michael@paquier.xyz>
escreveu:

On Wed, May 06, 2020 at 02:11:34PM -0400, Andrew Dunstan wrote:

We assume perl, flex and bison are in the PATH. That doesn't seem
unreasonable, it's worked well for quite a long time.

I recall that it is an assumption we rely on since MSVC scripts are
around, and that's rather easy to configure, so it seems to me that
changing things now would just introduce annoying changes for anybody
(developers, maintainers) using this stuff.

Ah yes, better to leave it as is. No problem for me, I already got around
the difficulty.

regards,
Ranier Vilela

On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:

Hacking pgbison.pl, to print PATH, shows that the path inside pgbison.pl,
returned to being the original, without the addition of c:\perl\bin;c:\bin.
my $out = $ENV{PATH};
print "Path after system call=$out\n";
Path after system
call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
The final part lacks: c:\perl\bin;c:\bin

Now I need to find out why the path is being reset, within the perl scripts.

FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
that uses Python, and that is now happy. One of my own machines uses
VS 2019 as well and I have yet to see what you are describing here.
Perhaps that's related to a difference in the version of perl you are
using and the version of that any others?
--
Michael

Em qua., 6 de mai. de 2020 às 21:14, Michael Paquier <michael@paquier.xyz>
escreveu:

On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:

Hacking pgbison.pl, to print PATH, shows that the path inside pgbison.pl

,

returned to being the original, without the addition of

c:\perl\bin;c:\bin.

my $out = $ENV{PATH};
print "Path after system call=$out\n";
Path after system
call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
The final part lacks: c:\perl\bin;c:\bin

Now I need to find out why the path is being reset, within the perl

scripts.

FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
that uses Python, and that is now happy. One of my own machines uses
VS 2019 as well and I have yet to see what you are describing here.
Perhaps that's related to a difference in the version of perl you are
using and the version of that any others?

I really don't know what to say, I know very little about perl.

The perl is:
Win32 strawberry-perl 5.30.1.1

regards,
Ranier VIlela

В Thu, 7 May 2020 09:14:33 +0900
Michael Paquier <michael@paquier.xyz> пишет:

On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:

Hacking pgbison.pl, to print PATH, shows that the path inside
pgbison.pl, returned to being the original, without the addition of
c:\perl\bin;c:\bin. my $out = $ENV{PATH};
print "Path after system call=$out\n";
Path after system
call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
The final part lacks: c:\perl\bin;c:\bin

Now I need to find out why the path is being reset, within the perl
scripts.

FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
that uses Python, and that is now happy. One of my own machines uses
VS 2019 as well and I have yet to see what you are describing here.
Perhaps that's related to a difference in the version of perl you are
using and the version of that any others?

I doubt so. I have different machines with perl from 5.22 to 5.30 but
none of tham exibits such weird behavoir.

Perhaps problem is that Ranier calls vcvars64.bat from the menu, and
then calls msbuild such way that is becames unrelated process.

Obvoisly buildfarm animal doesn't use menu and then starts build
process from same CMD.EXE process, that it called vcvarsall.but into.

It is same in all OSes - Windows, *nix and even MS-DOS - there is no
way to change environment of parent process. You can change environment
of current process (and if this process is command interpreter you can
do so by sourcing script into it. In windows this misleadingly called
'CALL', but it executes commands from command file in the current
shell, not in subshell) you can pass enivronment to the child
processes. But you can never affect environment of the parent or
sibling process.

The only exception is - if you know that some process would at startup
read environment vars from some file or registry, you can modify this
source in unrelated process.

So, if you want perl in path of msbuild, started from Visual Studio,
you should either first set path in CMD.EXE, then type command to start
Studio from this very command window, or set path via control panel
dialog (which modified registry). Later is what I usially do on machines
wher I compile postgres.

--

В Wed, 6 May 2020 21:23:57 -0300
Ranier Vilela <ranier.vf@gmail.com> пишет:

The perl is:
Win32 strawberry-perl 5.30.1.1

This perl would have problems when compiling PL/Perl (see my letter
about week ago), but it have no problems running various build scripts
for Postgres. I'm using it with MSVisualStudio 2019 and only unexpected
thing I've encountered is that it comes with its own patch.exe, which
doesn't like unix-style end-of-lines in patches (but OK with them in
patched files).

regards,
Ranier VIlela

--

On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha wrote:

Please forgive me if I am being too nitpicky, but I find the comments a
little too verbose, a usage format might be more visual and easier to
explain:

Usage: build [[CONFIGURATION] COMPONENT]

The options are case-insensitive.
CONFIGURATION sets the configuration to build, "debug" or "release" (by
default).
COMPONENT defines a component to build. An empty option means all
components.

Your comment makes sense to me. What about the attached then? On top
of documenting the script usage in the code, let's trigger it if it
gets called with more than 3 arguments. What do you think?

FWIW, I forgot to mention that I don't think those warnings are worth
a backpatch. No objections with improving things on HEAD of course.
--
Michael

Em qui., 7 de mai. de 2020 às 02:04, Victor Wagner <vitus@wagner.pp.ru>
escreveu:

В Thu, 7 May 2020 09:14:33 +0900
Michael Paquier <michael@paquier.xyz> пишет:

On Wed, May 06, 2020 at 03:58:15PM -0300, Ranier Vilela wrote:

Hacking pgbison.pl, to print PATH, shows that the path inside
pgbison.pl, returned to being the original, without the addition of
c:\perl\bin;c:\bin. my $out = $ENV{PATH};
print "Path after system call=$out\n";
Path after system
call=...C:\Users\ranier\AppData\Local\Microsoft\WindowsApps;;
The final part lacks: c:\perl\bin;c:\bin

Now I need to find out why the path is being reset, within the perl
scripts.

FWIW, we have a buildfarm animal called drongo that runs with VS 2019,
that uses Python, and that is now happy. One of my own machines uses
VS 2019 as well and I have yet to see what you are describing here.
Perhaps that's related to a difference in the version of perl you are
using and the version of that any others?

I doubt so. I have different machines with perl from 5.22 to 5.30 but
none of tham exibits such weird behavoir.

The perl is the same,when it was working ok.

Perhaps problem is that Ranier calls vcvars64.bat from the menu, and
then calls msbuild such way that is becames unrelated process.

It also worked previously, using this same process, link menu and manual
path configuration.
What has changed:
1 In the environment, the python installation, which added entries to the
path.
2. Perl scripts: Use perl's $/ more idiomatically
commit beb2516e961490723fb1a2f193406afb3d71ea9c
3. Msbuild and others, have been updated.They are not the same ones that
were working before.

Obvoisly buildfarm animal doesn't use menu and then starts build
process from same CMD.EXE process, that it called vcvarsall.but into.

It is same in all OSes - Windows, *nix and even MS-DOS - there is no
way to change environment of parent process. You can change environment
of current process (and if this process is command interpreter you can
do so by sourcing script into it. In windows this misleadingly called
'CALL', but it executes commands from command file in the current
shell, not in subshell) you can pass enivronment to the child
processes. But you can never affect environment of the parent or
sibling process.

Maybe that's what is happening, calling system, perl or msbuild, would be
creating a new environment, transferring the path that is configured in
Windows, and not the path that is in the environment that was manually
configured.

The only exception is - if you know that some process would at startup
read environment vars from some file or registry, you can modify this
source in unrelated process.

So, if you want perl in path of msbuild, started from Visual Studio,
you should either first set path in CMD.EXE, then type command to start
Studio from this very command window, or set path via control panel
dialog (which modified registry). Later is what I usially do on machines
wher I compile postgres.

buidfarm aninal, uses a more secure and reliable process, the path is
already configured and does not change.
Perhaps this is the way for me and for others.

It would then remain to document, to warn that to work correctly, the path
must be configured before entering the compilation environment.

regards,
Ranier Vilela

Hi Michael,

I performed a quick test for the path "msvc-build-init-v2.patch" using
below cases:

1. perl build.pl
2. perl build.pl debug psql
3. perl build.pl RELEASE psql
4. perl build.pl deBUG psql
5. perl build.pl psql
The above cases (case-insensitive) are all working great without any
warning for latest master branch.

When build with more than 3 parameters, yes, I got below expected
message as well.

c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl DEbug
psql pg_baseback
Usage: build.pl [ [ <configuration> ] <component> ]
Options are case-insensitive.
configuration: Release | Debug. This sets the configuration
to build. Default is Release.
component: name of component to build. An empty value means
to build all components.

However, if I ask for help in a typical Windows' way, i.e, "perl
build.pl -help" or "perl build.pl /?", I got some messages like below,

c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl -help
Detected hardware platform: Win32
Files src/bin/pgbench/exprscan.l
Files src/bin/pgbench/exprparse.y
Files src/bin/psql/psqlscanslash.l
Files contrib/cube/cubescan.l
Files contrib/cube/cubeparse.y
Files contrib/seg/segscan.l
Files contrib/seg/segparse.y
Generating configuration headers...
Microsoft (R) Build Engine version 16.5.1+4616136f8 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1001: Unknown switch.
Switch: -help.vcxproj

For switch syntax, type "MSBuild -help"

c:\Users\david\Downloads\postgres\src\tools\msvc>perl build.pl /?
Detected hardware platform: Win32
Files src/bin/pgbench/exprscan.l
Files src/bin/pgbench/exprparse.y
Files src/bin/psql/psqlscanslash.l
Files contrib/cube/cubescan.l
Files contrib/cube/cubeparse.y
Files contrib/seg/segscan.l
Files contrib/seg/segparse.y
Generating configuration headers...
Microsoft (R) Build Engine version 16.5.1+4616136f8 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

MSBUILD : error MSB1001: Unknown switch.
Switch: /?.vcxproj

For switch syntax, type "MSBuild -help"

It would be a bonus if the build.pl can support the "help" in Windows' way.

Thanks,

David

On 2020-05-06 10:45 p.m., Michael Paquier wrote:

On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha wrote:

Please forgive me if I am being too nitpicky, but I find the comments a
little too verbose, a usage format might be more visual and easier to
explain:

Usage: build [[CONFIGURATION] COMPONENT]

The options are case-insensitive.
CONFIGURATION sets the configuration to build, "debug" or "release" (by
default).
COMPONENT defines a component to build. An empty option means all
components.

Your comment makes sense to me. What about the attached then? On top
of documenting the script usage in the code, let's trigger it if it
gets called with more than 3 arguments. What do you think?

FWIW, I forgot to mention that I don't think those warnings are worth
a backpatch. No objections with improving things on HEAD of course.
--
Michael

--
David

Software Engineer
Highgo Software Inc. (Canada)
www.highgo.ca

On Tue, Jun 2, 2020 at 2:06 AM David Zhang <david.zhang@highgo.ca> wrote:

On 2020-05-06 10:45 p.m., Michael Paquier wrote:

On Wed, May 06, 2020 at 12:17:03AM +0200, Juan José Santamaría Flecha

wrote:

Please forgive me if I am being too nitpicky, but I find the comments a
little too verbose, a usage format might be more visual and easier to
explain:

Usage: build [[CONFIGURATION] COMPONENT]

The options are case-insensitive.
CONFIGURATION sets the configuration to build, "debug" or "release" (by
default).
COMPONENT defines a component to build. An empty option means all
components.

Your comment makes sense to me. What about the attached then? On top
of documenting the script usage in the code, let's trigger it if it
gets called with more than 3 arguments. What do you think?

FWIW, I forgot to mention that I don't think those warnings are worth
a backpatch. No objections with improving things on HEAD of course.

It would be a bonus if the build.pl can support the "help" in Windows'
way.

Going through the open items in the commitfest, I see that this patch has
not been pushed. It still applies and solves the warning so, I am marking
it as RFC.

Adding a help option is a new feature, that can have its own patch without
delaying this one any further.

Regards,

Juan José Santamaría Flecha

On Sat, Jul 04, 2020 at 09:17:52PM +0200, Juan José Santamaría Flecha wrote:

Going through the open items in the commitfest, I see that this patch has
not been pushed. It still applies and solves the warning so, I am marking
it as RFC.

Thanks, applied. I was actually waiting to see if you had more
comments.

Adding a help option is a new feature, that can have its own patch without
delaying this one any further.

Yep. And I am not sure if that's worth worrying either.
--
Michael