msvc and vista fun

Started by Andrew Dunstanover 18 years ago31 messages
#1Andrew Dunstan
andrew@dunslane.net

I am still very unhappy about the way the MSVC builds work. Although we
have managed to make it sort of work with the buildfarm script, it is
distinctly fragile. Last night, for example, I had a build failure due
to a badly installed zlib. The error state didn't come back to the
buildfarm script, which carried on merrily to the check stage, which
also naturally failed, but also didn't manage to give back an error
state to the buildfarm script, and finally got a failure at the install
stage. That means that we simply can't rely on this build system to give
us accurate state on the buildfarm - we might have had a code breakage
but it won't always tell us that.

Part of the problem, it seems to me, is that this build system is a
hodge-podge of Perl programs and Windows shell scripts. In at least one
case we call perl to write a dynamically generated .bat file which we
then execute. Since the perl is pretty much indispensable (or at least
would need replacing by an engine of similar capability), I think we
should make the whole thing a perl suite with some very minimal .bat
wrappers if necessary.

On a somewhat related note, I have had spectacular lack of success in
getting either MSVC or MinGW builds to work on Vista - so much so that I
have currently abandoned my attempts on that platform and I resorted to
resuscitating an old XP box for testing. Following some advice from
Magnus, I added ACLs to the build root for both an admin and a non-admin
user (cacls buildroot /E /T /G AdminUser:C and similarly for a non-admin
user) . I can build as the admin user but when I come to run initdb it
fails, complaining that it can't find the postgres executable. If I then
switch to the non-admin user, it can run initdb just fine. However, that
user can't build, because it gets a mysterious failure from mt.exe.
MinGW is even worse - it says it can't run gcc because it can't run
cc1.exe (IIRC), so it fails at the configure stage! All of this has cost
me quite a few hours in the last week, with very little to show for it.

Perhaps someone would like to tell me how I can remedy these problems.
More importantly, this should be in an FAQ or some such. Also, I would
like to know if we have really tested out on Vista the privilege
surrendering code that is is supposed to work in Windows. It looks to me
like it might not be working.

If we can make progress on these issues then I'll be happy. If not, I
think we need to look carefully at what we say we can support, and what
we declare to be still experimental.

cheers

andrew

#2Dave Page
dpage@postgresql.org
In reply to: Andrew Dunstan (#1)
Re: msvc and vista fun

Andrew Dunstan wrote:

On a somewhat related note, I have had spectacular lack of success in
getting either MSVC or MinGW builds to work on Vista - so much so that I
have currently abandoned my attempts on that platform and I resorted to
resuscitating an old XP box for testing. Following some advice from
Magnus, I added ACLs to the build root for both an admin and a non-admin
user (cacls buildroot /E /T /G AdminUser:C and similarly for a non-admin
user) . I can build as the admin user but when I come to run initdb it
fails, complaining that it can't find the postgres executable.

Yeah, I ran into that problem as well. I'll look at my Vista box when
I'm in the office tomorrow and see if I can figure out what hack fixed
it for me.

If I then

switch to the non-admin user, it can run initdb just fine. However, that
user can't build, because it gets a mysterious failure from mt.exe.
MinGW is even worse - it says it can't run gcc because it can't run
cc1.exe (IIRC), so it fails at the configure stage! All of this has cost
me quite a few hours in the last week, with very little to show for it.

And that one...

Perhaps someone would like to tell me how I can remedy these problems.
More importantly, this should be in an FAQ or some such. Also, I would
like to know if we have really tested out on Vista the privilege
surrendering code that is is supposed to work in Windows. It looks to me
like it might not be working.

What makes you say that?

If we can make progress on these issues then I'll be happy. If not, I
think we need to look carefully at what we say we can support, and what
we declare to be still experimental.

Lack of completely reliable buildfarm support for MSVC or trouble
setting up a buildenv doesn't mean we cannot support a platform - it
just means we need o be extra vigilent to ensure that regresssion tests
etc. do actually pass (though that level of failure would be visible on
the BF I would hope - it certainly has been until now).

Regards, Dave

#3Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#1)
Re: msvc and vista fun

Andrew Dunstan wrote:

I am still very unhappy about the way the MSVC builds work. Although we
have managed to make it sort of work with the buildfarm script, it is
distinctly fragile. Last night, for example, I had a build failure due
to a badly installed zlib. The error state didn't come back to the
buildfarm script, which carried on merrily to the check stage, which
also naturally failed, but also didn't manage to give back an error

I've heard you report this before, and I've tried and failed many times
to reproduce it. It has *always* come back with the proper errorlevel
when I've tried. So two questions:

1) are you absolutely sure that the problem is not in the buildfarm script?

2) What exactly was the error? So I can try to reproduce that exact
problem here and see if I can find out what it is. What did you do
wrong, and what was the error messages if you still have the log.

state to the buildfarm script, and finally got a failure at the install
stage. That means that we simply can't rely on this build system to give
us accurate state on the buildfarm - we might have had a code breakage
but it won't always tell us that.

At least it broke eventually :-) But I agree that's a big problem.

Part of the problem, it seems to me, is that this build system is a
hodge-podge of Perl programs and Windows shell scripts. In at least one
case we call perl to write a dynamically generated .bat file which we
then execute.

Which one is that? I can't find it. A simple findstr through all the
perl files appears to contain nothing of the sort.

Since the perl is pretty much indispensable (or at least
would need replacing by an engine of similar capability), I think we
should make the whole thing a perl suite with some very minimal .bat
wrappers if necessary.

That's mostly how it is now, no? What are you referring to specifically?

The only longer ones I can see are:

builddoc.bat - could certainly be put into perl stuff, but it's not
involved in the buildfarm process, so...

clean.bat - doesn't really matter, IMO, and again not involved in the
buildfarm process

vcregress.bat - sure, it could be made perl, but right now you can
actually run the regression tests on an installed system without having
perl at all... But that's not really a requirement, so we could change
that one.

The rest are just thin wrappers.

Speaking of wrappers, are you planning to make the buildfarm use the
modules instead of the perlscripts? IIRC, I made those modules
specifically for the bf - and it should make it even easier to pick up
the errors from there.

I'll leave the Vista parts to Dave since I haven't used it at all there.
These are really completely separate issues, and should be treated as such.

//Magnus

#4Andrew Dunstan
andrew@dunslane.net
In reply to: Magnus Hagander (#3)
Re: msvc and vista fun

Magnus Hagander wrote:

Andrew Dunstan wrote:

I am still very unhappy about the way the MSVC builds work. Although we
have managed to make it sort of work with the buildfarm script, it is
distinctly fragile. Last night, for example, I had a build failure due
to a badly installed zlib. The error state didn't come back to the
buildfarm script, which carried on merrily to the check stage, which
also naturally failed, but also didn't manage to give back an error

I've heard you report this before, and I've tried and failed many times
to reproduce it. It has *always* come back with the proper errorlevel
when I've tried. So two questions:

1) are you absolutely sure that the problem is not in the buildfarm script?

Pretty damn sure, yes. This code has functioned correctly on the
buildfarm on every other platform since its inception.

2) What exactly was the error? So I can try to reproduce that exact
problem here and see if I can find out what it is. What did you do
wrong, and what was the error messages if you still have the log.

I did indeed keep the logs.

Steps to reproduce: rename your zdll.lib and then run the buildfarm script.

Output:

E:\prog>perl run_build.pl --test
[17:51:47] checking out source ...
[17:52:17] checking if build run needed ...
[17:52:30] copying source to pgsql.3840 ...
3152 File(s) copied
[17:53:44] running configure ...
[17:53:44] running make ...
[18:27:50] running make check ...
[18:27:50] running make install ...
Branch: HEAD
Stage Install failed with status 2

from the bottom of the make log:

Build FAILED.
.\src\backend\utils\adt\oracle_compat.c(898): warning C4090: 'function'
: different 'const' qualifiers
.\src\backend\utils\adt\oracle_compat.c(900): warning C4090: 'function'
: different 'const' qualifiers
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
LINK : fatal error LNK1104: cannot open file
'e:\prog\pgdepend\zlib\lib\zdll.lib'
2 Warning(s)
12 Error(s)

Relevant perl code executed by buildfarm:

chdir "$pgsql/src/tools/msvc";
@makeout = `build 2>&1`;
chdir $branch_root;
my $status = $? >>8;

Could it possibly be that the exit status has only low bits set, and
isn't what should be returned by wait()? I am checking on this now ....
(time passes) ... no, the raw status returned is in fact 0.

Now, I see that we do something different in the install case, which is
why the error gets trapped there. For install, we don't call the .bat
file at all, we call the install script directly:

chdir "$pgsql/src/tools/msvc";
@makeout = `perl install.pl "$installdir" 2>&1`;
chdir $branch_root;
my $status = $? >>8;

So it looks like the .bat files are not collecting the exit status and
passing it on correctly.

state to the buildfarm script, and finally got a failure at the install
stage. That means that we simply can't rely on this build system to give
us accurate state on the buildfarm - we might have had a code breakage
but it won't always tell us that.

At least it broke eventually :-) But I agree that's a big problem.

yes.

Part of the problem, it seems to me, is that this build system is a
hodge-podge of Perl programs and Windows shell scripts. In at least one
case we call perl to write a dynamically generated .bat file which we
then execute.

Which one is that? I can't find it. A simple findstr through all the
perl files appears to contain nothing of the sort.

perl ../../src/tools/msvc/getregress.pl > regress.tmp.bat
call regress.tmp.bat
del regress.tmp.bat

Since the perl is pretty much indispensable (or at least
would need replacing by an engine of similar capability), I think we
should make the whole thing a perl suite with some very minimal .bat
wrappers if necessary.

That's mostly how it is now, no? What are you referring to specifically?

The only longer ones I can see are:

builddoc.bat - could certainly be put into perl stuff, but it's not
involved in the buildfarm process, so...

clean.bat - doesn't really matter, IMO, and again not involved in the
buildfarm process

vcregress.bat - sure, it could be made perl, but right now you can
actually run the regression tests on an installed system without having
perl at all... But that's not really a requirement, so we could change
that one.

perl2exe ?

The rest are just thin wrappers.

Well, my .bat-fu was last in use around DOS6.2, so I'm not mucjh help
with them :-) I don't think they're really thin enough. Why not just
have them call "perl scriptname args"?

Speaking of wrappers, are you planning to make the buildfarm use the
modules instead of the perlscripts? IIRC, I made those modules
specifically for the bf - and it should make it even easier to pick up
the errors from there.

Yes, you did. But as I think I told you, I had a rethink about that. The
whole point about the buildfarm is to do things as closely to what a
human would do as possible, so I don't want to make it use the perl
modules like that unless we really really have to. I also want to have
as little special casing as possible in the buildfarm.

cheers

andrew

#5Andrew Dunstan
andrew@dunslane.net
In reply to: Dave Page (#2)
Re: msvc and vista fun

Dave Page wrote:

Perhaps someone would like to tell me how I can remedy these
problems. More importantly, this should be in an FAQ or some such.
Also, I would like to know if we have really tested out on Vista the
privilege surrendering code that is is supposed to work in Windows.
It looks to me like it might not be working.

What makes you say that?

It was just a suspicion, but maybe the fact that the admin user can't
run while the non-admin user can indicates that it is working (possibly
too well).

If we can make progress on these issues then I'll be happy. If not, I
think we need to look carefully at what we say we can support, and
what we declare to be still experimental.

Lack of completely reliable buildfarm support for MSVC or trouble
setting up a buildenv doesn't mean we cannot support a platform - it
just means we need o be extra vigilent to ensure that regresssion
tests etc. do actually pass (though that level of failure would be
visible on the BF I would hope - it certainly has been until now).

I am not only concerned about the buildfarm, but about the robustness,
maintainability and simplicity of the build process. Having to have a
completely separate build system for msvc is really quite sucky.

cheers

andrew

#6Michael Paesold
mpaesold@gmx.at
In reply to: Andrew Dunstan (#4)
Re: msvc and vista fun

Andrew Dunstan wrote:

Relevant perl code executed by buildfarm:

chdir "$pgsql/src/tools/msvc";
@makeout = `build 2>&1`;
chdir $branch_root;
my $status = $? >>8;

I know the docs say otherwise, but would it be possible that "chdir"
somehow resets $? on windows, sometimes, under some circumstances?

Perhaps just move up the "my $status.." one line up?

Best Regards
Michael Paesold

#7Zeugswetter Andreas ADI SD
Andreas.Zeugswetter@s-itsolutions.at
In reply to: Andrew Dunstan (#1)
Re: msvc and vista fun

user) . I can build as the admin user but when I come to run
initdb it fails, complaining that it can't find the postgres
executable.

FYI, this happens on my Win 2000 also.
Maybe a problem with mixed / \ path separators after RestrictExec.

Andreas

#8Dave Page
dpage@postgresql.org
In reply to: Andrew Dunstan (#5)
Re: msvc and vista fun

Andrew Dunstan wrote:

Dave Page wrote:

Perhaps someone would like to tell me how I can remedy these
problems. More importantly, this should be in an FAQ or some such.
Also, I would like to know if we have really tested out on Vista the
privilege surrendering code that is is supposed to work in Windows.
It looks to me like it might not be working.

What makes you say that?

It was just a suspicion, but maybe the fact that the admin user can't
run while the non-admin user can indicates that it is working (possibly
too well).

Hmm, yes - I see what you mean. How are you defining 'admin user'? When
I ran into this problem from what I recall I was originally trying to
swim against the Vista security model by using the Administrator account
which is disabled by default (for anyone still doubting that from the
last discussion on the topic, please see
http://blogs.msdn.com/windowsvistasecurity/archive/2006/08/27/windowsvistasecurity-.aspx).
It went away when I started using my own user account which is not 'the'
administrator, but is a member of the administrators group.

I am not only concerned about the buildfarm, but about the robustness,
maintainability and simplicity of the build process. Having to have a
completely separate build system for msvc is really quite sucky.

Well, yes, but unfortunately there's not really any other way to get a
halfway decent development environment on Windows. Mingw/Msys are
horrendously maintained - mingw still doesn't work without hacking on
Vista as you've found, and msys doesn't work at all on 64 bit versions
for example (at least it didn't the last few times I tried - it's
possible they've fixed it now). The debugger doesn't work for most
people, and the development environment is just plain nasty to keep up
to date.

Now 99.99% of users don't build on Windows anyway, so I'm not so much
concerned about their ability to maintain and use the build environment
as ours - if that means we have to fix a few bugs in the MSVC scripts
during the dev cycle, so be it. The setup I'm using now for 8.3 is
already orders of magnitude easier to setup and maintain than the
mingw/msys environment, plus we have a decent debugger (and other tools)
now, and the ability to run on 64 bit platforms, not to mention a
compiler with which we can (eventually) build for them as well.

Regards, Dave

#9Dave Page
dpage@postgresql.org
In reply to: Dave Page (#2)
Re: msvc and vista fun

Dave Page wrote:

If I then

switch to the non-admin user, it can run initdb just fine. However,
that user can't build, because it gets a mysterious failure from
mt.exe. MinGW is even worse - it says it can't run gcc because it
can't run cc1.exe (IIRC), so it fails at the configure stage! All of
this has cost me quite a few hours in the last week, with very little
to show for it.

And that one...

See http://aarongiles.com/?p=199 for more details, but the quick fix I
used was to add /mingw/libexec/gcc/mingw32/3.4.2 to the path in
/etc/profile.

Regards, Dave

#10Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#4)
Re: msvc and vista fun

On Sun, Jun 24, 2007 at 08:24:43PM -0400, Andrew Dunstan wrote:

Magnus Hagander wrote:

Andrew Dunstan wrote:

I am still very unhappy about the way the MSVC builds work. Although we
have managed to make it sort of work with the buildfarm script, it is
distinctly fragile. Last night, for example, I had a build failure due
to a badly installed zlib. The error state didn't come back to the
buildfarm script, which carried on merrily to the check stage, which
also naturally failed, but also didn't manage to give back an error

I've heard you report this before, and I've tried and failed many times
to reproduce it. It has *always* come back with the proper errorlevel
when I've tried. So two questions:

1) are you absolutely sure that the problem is not in the buildfarm script?

Pretty damn sure, yes. This code has functioned correctly on the
buildfarm on every other platform since its inception.

I think you are overestimating the cross-platformness of perl... :-)

2) What exactly was the error? So I can try to reproduce that exact
problem here and see if I can find out what it is. What did you do
wrong, and what was the error messages if you still have the log.

I did indeed keep the logs.

Steps to reproduce: rename your zdll.lib and then run the buildfarm script.

Ok.

Running this *outside* the buildfarm properly sets errorlevel=1 when it
exits.

Running the same test from the buildfarm script on the same machine, and it
picks up the error and reports it just fine.
(http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=skylark&dt=2007-06-25%2008:28:31)

Since nobody else (AFAIK) has seen this, I'd be inclined to blame something
in your environment.

Can you run the build in "broken mode" but *without* using the buildfarm
scripts, and see what errorlevel you get? Meaning:
build.bat
... wait ...
echo %errorlevel%

It should output 1 if that part works, 0 if it fails. That'll tell us if
the problem is in the bf or if it's in the actual build system.

Part of the problem, it seems to me, is that this build system is a
hodge-podge of Perl programs and Windows shell scripts. In at least one
case we call perl to write a dynamically generated .bat file which we
then execute.

Which one is that? I can't find it. A simple findstr through all the
perl files appears to contain nothing of the sort.

perl ../../src/tools/msvc/getregress.pl > regress.tmp.bat
call regress.tmp.bat
del regress.tmp.bat

Ah. I was looking at the build parts only, not the regress part. My bad.

Anyway, yes, the vcregress.bat file was the only one on the list that made
sense to consider redoing in perl.

Since the perl is pretty much indispensable (or at least
would need replacing by an engine of similar capability), I think we
should make the whole thing a perl suite with some very minimal .bat
wrappers if necessary.

That's mostly how it is now, no? What are you referring to specifically?

The only longer ones I can see are:

builddoc.bat - could certainly be put into perl stuff, but it's not
involved in the buildfarm process, so...

clean.bat - doesn't really matter, IMO, and again not involved in the
buildfarm process

vcregress.bat - sure, it could be made perl, but right now you can
actually run the regression tests on an installed system without having
perl at all... But that's not really a requirement, so we could change
that one.

perl2exe ?

No way, that's just too ugly :-) Since you basically ship perl packaged
inside the exe anyway, we might as well accept the requirement of perl to
run the regression tests. Actually, �given the above we *already* require
perl for the regression tests, I had forgotten about that part.

The rest are just thin wrappers.

Well, my .bat-fu was last in use around DOS6.2, so I'm not mucjh help
with them :-) I don't think they're really thin enough. Why not just
have them call "perl scriptname args"?

I guess you could. The point of the split in build.bat today is basically
that you run mkvcbuild separately if you want to work from the GUI. But
that could of course be dealt with with commandline parameters if there's
really a benefit in it. But I'm not sure there is - there's very little
logic in the bat files, and that's what matters IMO.

Speaking of wrappers, are you planning to make the buildfarm use the
modules instead of the perlscripts? IIRC, I made those modules
specifically for the bf - and it should make it even easier to pick up
the errors from there.

Yes, you did. But as I think I told you, I had a rethink about that. The
whole point about the buildfarm is to do things as closely to what a
human would do as possible, so I don't want to make it use the perl
modules like that unless we really really have to. I also want to have
as little special casing as possible in the buildfarm.

Ah, you did tell me that. Sorry, forgot about that. Fair 'nuff.

//Magnus

#11Dave Page
dpage@postgresql.org
In reply to: Magnus Hagander (#10)
Re: msvc and vista fun

Magnus Hagander wrote:

Running the same test from the buildfarm script on the same machine, and it
picks up the error and reports it just fine.
(http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=skylark&dt=2007-06-25%2008:28:31)

I ran on Baiji (renaming the zlib directory, instead of the .lib as
Magnus did) and see a error as well:
http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=baiji&dt=2007-06-25%2008:36:47

Just for kicks, that was also on Vista. I'm going to run on my Win2K3
animal in a minute - that one runs VC++ Express, unlike Baiji and
Skylark which I believe are both 'proper' Visual Studio. Will report
back when it's done.

Regards, Dave

#12Dave Page
dpage@postgresql.org
In reply to: Dave Page (#11)
Re: msvc and vista fun

Dave Page wrote:

I'm going to run on my Win2K3
animal in a minute - that one runs VC++ Express, unlike Baiji and
Skylark which I believe are both 'proper' Visual Studio. Will report
back when it's done.

Yep, it failed at make as well, and reported it appropriately to the BF.

Regards, Dave

#13Andrew Dunstan
andrew@dunslane.net
In reply to: Dave Page (#12)
Re: msvc and vista fun

Dave Page wrote:

Dave Page wrote:

I'm going to run on my Win2K3 animal in a minute - that one runs VC++
Express, unlike Baiji and Skylark which I believe are both 'proper'
Visual Studio. Will report back when it's done.

Yep, it failed at make as well, and reported it appropriately to the BF.

Well, that's *very* curious. I have seen this on more than one machine.

I'll run the test Magnus suggests and see what we can discover.

cheers

andrew

#14Andrew Dunstan
andrew@dunslane.net
In reply to: Magnus Hagander (#10)
Re: msvc and vista fun

Magnus Hagander wrote:

Can you run the build in "broken mode" but *without* using the buildfarm
scripts, and see what errorlevel you get? Meaning:
build.bat
... wait ...
echo %errorlevel%

It should output 1 if that part works, 0 if it fails. That'll tell us if
the problem is in the bf or if it's in the actual build system.

It fails and says 1.

Now, if I change the build script so it says

exit %E

instead of

exit /b %E

I pick up the exit status just fine. I wonder if wer have different
flavors of command interpreter? (My perl is the latest one from
ActiveState, btw. I assume you're using that.)

Can we change that or make it switchable? I'd be happy to provide an
environment variable like RUNNING_BUILDFARM if that would help.

cheers

andrew

#15Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#14)
Re: msvc and vista fun

On Mon, Jun 25, 2007 at 10:06:57AM -0400, Andrew Dunstan wrote:

Magnus Hagander wrote:

Can you run the build in "broken mode" but *without* using the buildfarm
scripts, and see what errorlevel you get? Meaning:
build.bat
... wait ...
echo %errorlevel%

It should output 1 if that part works, 0 if it fails. That'll tell us if
the problem is in the bf or if it's in the actual build system.

It fails and says 1.

Now, if I change the build script so it says

exit %E

instead of

exit /b %E

That's just weird :-( And making that change makes the script unusable from
the commandline.

I pick up the exit status just fine. I wonder if wer have different
flavors of command interpreter? (My perl is the latest one from
ActiveState, btw. I assume you're using that.)

cmd.exe from Windows - I assume you haven't installed some funky addon
special command interpreter? In that case, it should be the same. So I
have:
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

As for perl, I'm probably not on the very latest, but it's not so old. I'm
on:
This is perl, v5.8.8 built for MSWin32-x64-multi-thread
(with 33 registered patches, see perl -V for more detail)

Dave - what version are you on?

Can we change that or make it switchable? I'd be happy to provide an
environment variable like RUNNING_BUILDFARM if that would help.

We could, but it seems very ugly. And again, it's *not* required for
buildfarm on my or Daves machines. So I'd rather like to know why it's
actually happening than just blindly change it.

//Magnus

#16Dave Page
dpage@postgresql.org
In reply to: Magnus Hagander (#15)
Re: msvc and vista fun

Magnus Hagander wrote:

As for perl, I'm probably not on the very latest, but it's not so old. I'm
on:
This is perl, v5.8.8 built for MSWin32-x64-multi-thread
(with 33 registered patches, see perl -V for more detail)

Dave - what version are you on?

This is perl, v5.8.8 built for MSWin32-x86-multi-thread
(with 50 registered patches, see perl -V for more detail)

/D

#17Andrew Dunstan
andrew@dunslane.net
In reply to: Magnus Hagander (#15)
Re: msvc and vista fun

Magnus Hagander wrote:

cmd.exe from Windows - I assume you haven't installed some funky addon
special command interpreter? In that case, it should be the same. So I
have:
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

Hmm. Mine says 5.1.2600, which seems to be quite old. I'll see about
updating ... the strange thing is I'm pretty sure I've seen this on
other more up to date machines.

cheers

andrew

#18Dave Page
dpage@postgresql.org
In reply to: Andrew Dunstan (#17)
Re: msvc and vista fun

Andrew Dunstan wrote:

Hmm. Mine says 5.1.2600, which seems to be quite old. I'll see about
updating ... the strange thing is I'm pretty sure I've seen this on
other more up to date machines.

You can't update that unless you upgrade to Windows 2003 or Vista.
That's the XP command shell.

Regards, Dave

#19Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#17)
Re: msvc and vista fun

On Mon, Jun 25, 2007 at 10:45:00AM -0400, Andrew Dunstan wrote:

Magnus Hagander wrote:

cmd.exe from Windows - I assume you haven't installed some funky addon
special command interpreter? In that case, it should be the same. So I
have:
Microsoft Windows [Version 5.2.3790]
(C) Copyright 1985-2003 Microsoft Corp.

Hmm. Mine says 5.1.2600, which seems to be quite old. I'll see about
updating ... the strange thing is I'm pretty sure I've seen this on
other more up to date machines.

That's because you are on XP. 5.1 is XP, 5.2 is 2003. To make things more
interesting, XP x64 is 5.2...

So it might be worthwhile to see if this is something that happens on <=
XP, and works on >= 2003. Dave, did you test on any server OS other than
2003, or any client that's XP or earlier?

//Magnus

#20Andrew Dunstan
andrew@dunslane.net
In reply to: Dave Page (#9)
Re: msvc and vista fun

Dave Page wrote:

Dave Page wrote:

If I then

switch to the non-admin user, it can run initdb just fine. However,
that user can't build, because it gets a mysterious failure from
mt.exe. MinGW is even worse - it says it can't run gcc because it
can't run cc1.exe (IIRC), so it fails at the configure stage! All of
this has cost me quite a few hours in the last week, with very
little to show for it.

And that one...

See http://aarongiles.com/?p=199 for more details, but the quick fix I
used was to add /mingw/libexec/gcc/mingw32/3.4.2 to the path in
/etc/profile.

That seems to have fixed it. But now of cousre I forgot that the latest
version is broken w.r.t. gettimeofday ... *sigh*

cheers

andrew

#21Andrew Dunstan
andrew@dunslane.net
In reply to: Magnus Hagander (#15)
Re: msvc and vista fun

Magnus Hagander wrote:

Can we change that or make it switchable? I'd be happy to provide an
environment variable like RUNNING_BUILDFARM if that would help.

We could, but it seems very ugly. And again, it's *not* required for
buildfarm on my or Daves machines. So I'd rather like to know why it's
actually happening than just blindly change it.

Not surprisingly, I am not the only person who has had this problem. See
http://jira.codehaus.org/browse/CONTINUUM-413 where the suggested
solution is:

if "%MAVEN_TERMINATE_CMD%" == "on" exit %ERRORLEVEL%
exit /b %ERRORLEVEL%

There are 12 lines involved in our .bat files:

build.bat:35:exit /b %E%
builddoc.bat:40:exit /b
builddoc.bat:50:exit /b
install.bat:9:exit /b 1
install.bat:18:exit /b %ERRORLEVEL%
vcregress.bat:35: if errorlevel 1 exit /b 1
vcregress.bat:51:exit /b %E%
vcregress.bat:64: if errorlevel 1 exit /b 1
vcregress.bat:66: if errorlevel 1 exit /b 1
vcregress.bat:86:exit /b %E%
vcregress.bat:97:if %CONTRIBERROR%==1 exit /b 1
vcregress.bat:113:exit /b %E%

Would making a change like this in those 12 places be so ugly?

cheers

andrew

#22Dave Page
dpage@postgresql.org
In reply to: Magnus Hagander (#19)
Re: msvc and vista fun

Magnus Hagander wrote:

So it might be worthwhile to see if this is something that happens on <=
XP, and works on >= 2003. Dave, did you test on any server OS other than
2003, or any client that's XP or earlier?

No, my testing was only on 2k3 and vista.

/D

#23Mark Cave-Ayland
mark.cave-ayland@ilande.co.uk
In reply to: Andrew Dunstan (#1)
Re: msvc and vista fun

On Sun, 2007-06-24 at 13:23 -0400, Andrew Dunstan wrote:

(cut)

On a somewhat related note, I have had spectacular lack of success in
getting either MSVC or MinGW builds to work on Vista - so much so that I
have currently abandoned my attempts on that platform and I resorted to
resuscitating an old XP box for testing. Following some advice from
Magnus, I added ACLs to the build root for both an admin and a non-admin
user (cacls buildroot /E /T /G AdminUser:C and similarly for a non-admin
user) . I can build as the admin user but when I come to run initdb it
fails, complaining that it can't find the postgres executable. If I then
switch to the non-admin user, it can run initdb just fine. However, that
user can't build, because it gets a mysterious failure from mt.exe.
MinGW is even worse - it says it can't run gcc because it can't run
cc1.exe (IIRC), so it fails at the configure stage! All of this has cost
me quite a few hours in the last week, with very little to show for it.

Hi Andrew,

This has come up quite recently on the MingW lists - see
http://sourceforge.net/mailarchive/forum.php?thread_name=00fc01c7a0af%
240a037c30%240200a8c0%40AMD2500&forum_name=mingw-users for a discussion
of the problem and solution.

Kind regards,

Mark.

--
ILande - Open Source Consultancy
http://www.ilande.co.uk

#24Andrew Dunstan
andrew@dunslane.net
In reply to: Mark Cave-Ayland (#23)
Re: msvc and vista fun

Mark Cave-Ayland wrote:

This has come up quite recently on the MingW lists - see
http://sourceforge.net/mailarchive/forum.php?thread_name=00fc01c7a0af%
240a037c30%240200a8c0%40AMD2500&forum_name=mingw-users for a discussion
of the problem and solution.

Thanks. I've actually got past this issue, and moved on to the problem
of having to downgrade far enough to get past the gettimeofday
redefinition problem :-)

cheers

andrew

#25Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#1)
Re: [HACKERS] msvc and vista fun

Andrew Dunstan wrote:

I wrote:

Would making a change like this in those 12 places be so ugly?

Specifically, I propose the following patch, which should fix the issues
buildfarm apparently has with the XP command shell (or some incarnations
of it).

Strictly speaking, we don't need the changes in builddoc.bat and
install.bat, but I did them for the sake of completeness. Since it
should not affect anyone not setting XP_EXIT_FIX it seems fairly low risk.

Yeah.
I still would like to have it confirmed that this is actually an XP
fault, and not something else in your environment. To that end, I've
bugged Dave to run some builds on his XP VM (since it apparently works
fine on my XP x64 machine).

//magnus

#26Andrew Dunstan
andrew@dunslane.net
In reply to: Magnus Hagander (#25)
Re: [HACKERS] msvc and vista fun

Magnus Hagander wrote:

Andrew Dunstan wrote:

I wrote:

Would making a change like this in those 12 places be so ugly?

Specifically, I propose the following patch, which should fix the issues
buildfarm apparently has with the XP command shell (or some incarnations
of it).

Strictly speaking, we don't need the changes in builddoc.bat and
install.bat, but I did them for the sake of completeness. Since it
should not affect anyone not setting XP_EXIT_FIX it seems fairly low risk.

Yeah.
I still would like to have it confirmed that this is actually an XP
fault, and not something else in your environment. To that end, I've
bugged Dave to run some builds on his XP VM (since it apparently works
fine on my XP x64 machine).

Ok. FWIW, this box is extremely clean and up to date. And I had the
identical issue on my previous XP/x386 box. It would be very strange if
somehow two environments got polluted like this.

cheers

andrew

#27Magnus Hagander
magnus@hagander.net
In reply to: Andrew Dunstan (#1)
Re: [HACKERS] msvc and vista fun

On Mon, Jun 25, 2007 at 03:19:32PM -0400, Andrew Dunstan wrote:

I wrote:

Would making a change like this in those 12 places be so ugly?

Specifically, I propose the following patch, which should fix the issues
buildfarm apparently has with the XP command shell (or some incarnations
of it).

Yeah. Dave has verified taht it's actually because of the XP stuff (he can
repro the failure on XP), so I guess we don't have much choice.

Strictly speaking, we don't need the changes in builddoc.bat and
install.bat, but I did them for the sake of completeness. Since it
should not affect anyone not setting XP_EXIT_FIX it seems fairly low risk.

I've applied it, except for builddoc.bat. Since we didn't send out an
exitcode anyway there, it makes no difference. Left them in install.bat
because there's an actual exitcode there.

//Magnus

#28Andrew Dunstan
andrew@dunslane.net
In reply to: Dave Page (#2)
Re: msvc and vista fun

Dave Page wrote:

Andrew Dunstan wrote:

On a somewhat related note, I have had spectacular lack of success in
getting either MSVC or MinGW builds to work on Vista - so much so
that I have currently abandoned my attempts on that platform and I
resorted to resuscitating an old XP box for testing. Following some
advice from Magnus, I added ACLs to the build root for both an admin
and a non-admin user (cacls buildroot /E /T /G AdminUser:C and
similarly for a non-admin user) . I can build as the admin user but
when I come to run initdb it fails, complaining that it can't find
the postgres executable.

Yeah, I ran into that problem as well. I'll look at my Vista box when
I'm in the office tomorrow and see if I can figure out what hack fixed
it for me.

I have never heard back on this, AFAIK. If anyone has instructions on
how to manage this please let me know. My current status with MSVC/vista
is still that I can build but not run as an admin user, and run but not
build as a non-admin user. Bleah.

cheers

andrew

#29Andrei Kovalevski
andyk@commandprompt.com
In reply to: Andrew Dunstan (#28)
Re: msvc and vista fun

Andrew Dunstan wrote:

Dave Page wrote:

Andrew Dunstan wrote:

On a somewhat related note, I have had spectacular lack of success
in getting either MSVC or MinGW builds to work on Vista - so much so
that I have currently abandoned my attempts on that platform and I
resorted to resuscitating an old XP box for testing. Following some
advice from Magnus, I added ACLs to the build root for both an admin
and a non-admin user (cacls buildroot /E /T /G AdminUser:C and
similarly for a non-admin user) . I can build as the admin user but
when I come to run initdb it fails, complaining that it can't find
the postgres executable.

Yeah, I ran into that problem as well. I'll look at my Vista box when
I'm in the office tomorrow and see if I can figure out what hack
fixed it for me.

I have never heard back on this, AFAIK. If anyone has instructions on
how to manage this please let me know. My current status with
MSVC/vista is still that I can build but not run as an admin user, and
run but not build as a non-admin user. Bleah.

cheers

andrew

Described situation looks like you are trying to run initdb under Admin
account. This will happen even if currently logged user in not admin,
but initdb has property 'run as administrator' set or you are trying to
run it under some file commander which is running in admin mode.

Andrei.

#30Andrew Dunstan
andrew@dunslane.net
In reply to: Andrei Kovalevski (#29)
Re: msvc and vista fun

Andrei Kovalevski wrote:

Andrew Dunstan wrote:

Dave Page wrote:

Andrew Dunstan wrote:

On a somewhat related note, I have had spectacular lack of success
in getting either MSVC or MinGW builds to work on Vista - so much
so that I have currently abandoned my attempts on that platform and
I resorted to resuscitating an old XP box for testing. Following
some advice from Magnus, I added ACLs to the build root for both an
admin and a non-admin user (cacls buildroot /E /T /G AdminUser:C
and similarly for a non-admin user) . I can build as the admin user
but when I come to run initdb it fails, complaining that it can't
find the postgres executable.

Yeah, I ran into that problem as well. I'll look at my Vista box
when I'm in the office tomorrow and see if I can figure out what
hack fixed it for me.

I have never heard back on this, AFAIK. If anyone has instructions on
how to manage this please let me know. My current status with
MSVC/vista is still that I can build but not run as an admin user,
and run but not build as a non-admin user. Bleah.

cheers

andrew

Described situation looks like you are trying to run initdb under
Admin account. This will happen even if currently logged user in not
admin, but initdb has property 'run as administrator' set or you are
trying to run it under some file commander which is running in admin
mode.

No it doesn't. Please read again. As a non-admin user I *can* run. I
just can't build. As an admin user I can build, but I can't run (and I
should be able to run).

cheers

andrew

#31Dave Page
dpage@postgresql.org
In reply to: Andrew Dunstan (#28)
Re: msvc and vista fun

Andrew Dunstan wrote:

I have never heard back on this, AFAIK. If anyone has instructions on
how to manage this please let me know. My current status with MSVC/vista
is still that I can build but not run as an admin user, and run but not
build as a non-admin user. Bleah.

I remember responding, but we had 2 somewhat intertwined threads going
back then as I recall so it might have been in the other one...

On my machine here, I run the buildfarm under user 'Dave' which is a
member of the Administrators group. It would not run under the
(unlocked) Administrator account as I recall, as it gave the error you
report about not being able to find postgres.exe. iirc, this is because
of the way the administrator account is largely crippled by UAC these
days (you're not really supposed to use it anyway).

I run the clients for both MSVC and Mingw this way. On Mingw, the only
other problem I recall was that cc1.exe couldn't be found, for which I
posted a solution here
http://archives.postgresql.org/pgsql-hackers/2007-06/msg00932.php

On MSVC, I don't recall seeing the problem you get with mt.exe. Can you
provide any more useful detail than 'mysterious failure'? That was all
you said in your original message.

Regards, Dave.