Building with Visual C++

"Magnus Hagander" <mha@sollentuna.net> writes:

*) Change where NaN is generated. Using 0.0/0.0 doesn't work in VC++
because it detects it as a divide-by-zero at the compile stage.

If they're going to be that anally uncooperative, why don't they have
the required-by-C99-spec macro for NAN? Or at least some well-defined
way to generate a NaN?

regards, tom lane

""Magnus Hagander"" <mha@sollentuna.net> wrote

I've been working on getting the full backend to compile and run using
Visual C++ instead of mingw/gcc, and have made some good progress.

This is great!

*) Add s_lock implementation based on InterlockedCompareExchange instead
of assembly code (gcc style asm not supported, and this should probably
be about as fast from what I hear)

_InterlockedCompareExchange is even better IMHO ...

src/port/dirent.c - opendir/readdir/closedir implementation for win32,
that doesn't even have the bugs mingw had in it ;)

But seems it has a typo ;-) Check out this line:

errno = GetLastError();

Regards,
Qingqing

I've been working on getting the full backend to compile

and run using

Visual C++ instead of mingw/gcc, and have made some good progress.

This is great!

Thanks.

*) Add s_lock implementation based on InterlockedCompareExchange
instead of assembly code (gcc style asm not supported, and

this should

probably be about as fast from what I hear)

_InterlockedCompareExchange is even better IMHO ...

I could've sworn the non-_ was #defined to the other one, but
double-checkign that I see that's only done on 64-bit systems. Will
update.

src/port/dirent.c - opendir/readdir/closedir implementation

for win32,

that doesn't even have the bugs mingw had in it ;)

But seems it has a typo ;-) Check out this line:

errno = GetLastError();

Argh. That was one point where I was lazy.

The solution to that is of course to use _dosmaperr(). The problem is
that _dosmaperr is in backend/port/error.c, but dirent is needed both in
frontend and backend.
To fix this, I suggest we move _dosmaperr() into src/port instead.
Probably renaming the file to win32error.c in the process. (The other
way would be to explicitly reference that file from the utils that need
dirent, but that just seems way ugly)
Thoughts?

//Magnus

*) Change where NaN is generated. Using 0.0/0.0 doesn't

work in VC++

because it detects it as a divide-by-zero at the compile stage.

If they're going to be that anally uncooperative, why don't
they have the required-by-C99-spec macro for NAN? Or at
least some well-defined way to generate a NaN?

They do have one way that's documented on MSDN, which is:
unsigned long nan[2]={0xffffffff, 0x7fffffff};
double g = *( double* )nan;

I thought that was even uglier ;-), but I can change it to use that on
win32 if you prefer it?

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

If they're going to be that anally uncooperative, why don't
they have the required-by-C99-spec macro for NAN? Or at
least some well-defined way to generate a NaN?

They do have one way that's documented on MSDN, which is:
unsigned long nan[2]={0xffffffff, 0x7fffffff};
double g = *( double* )nan;

I thought that was even uglier ;-), but I can change it to use that on
win32 if you prefer it?

Count on MSFT to violate the spec and be incredibly ugly and unportable
all at the same time.

How about

#if defined(WIN32) && !defined(NAN)
static const uint32 nan[2] ...
#define NAN (*(const double *) nan)
#endif

somewhere near the top of float.c (after including <math.h>, which is
what's supposed to define NAN). There doesn't seem to be much we can do
about the endianness assumption in their hack, but at least we can avoid
the assumption about sizeof(long).

regards, tom lane

*) Add s_lock implementation based on InterlockedCompareExchange
instead of assembly code (gcc style asm not supported, and

this should

probably be about as fast from what I hear)

_InterlockedCompareExchange is even better IMHO ...

I could've sworn the non-_ was #defined to the other one, but
double-checkign that I see that's only done on 64-bit
systems. Will update.

Hm. Wrong again :-(
Seems the whole _InterlockedCompareExchange version is only *available*
on 64-bit.

So we might want to look into doing that thing in assembly anyway, but
for now I'll just stick with the standard InterlockedCompareExchange.

//Magnus

If they're going to be that anally uncooperative, why

don't they have

the required-by-C99-spec macro for NAN? Or at least some
well-defined way to generate a NaN?

They do have one way that's documented on MSDN, which is:
unsigned long nan[2]={0xffffffff, 0x7fffffff}; double g =

*( double*

)nan;

I thought that was even uglier ;-), but I can change it to

use that on

win32 if you prefer it?

Count on MSFT to violate the spec and be incredibly ugly and
unportable all at the same time.

How about

#if defined(WIN32) && !defined(NAN)
static const uint32 nan[2] ...
#define NAN (*(const double *) nan)
#endif

somewhere near the top of float.c (after including <math.h>,
which is what's supposed to define NAN). There doesn't seem
to be much we can do about the endianness assumption in their
hack, but at least we can avoid the assumption about sizeof(long).

That worked. Endianness shouldn't be a big problem, because win32
doesn't run on more than one endianness....

I'll incorporate this in a new version of the patch - just need to
figure out what to do about _dosmaperror() and the cbrt() issue as well
:-)

//Magnus

I've done a similar thing myself (building the complete backend with
Visual Studio 2005). I got it to basically work, but stopped due to
lack of interest in the PostgreSQL community.

But, I'd be happy to help out your effort in any way I can.

-----Original Message-----
From: pgsql-patches-owner@postgresql.org [mailto:pgsql-patches-
owner@postgresql.org] On Behalf Of Magnus Hagander
Sent: Sunday, April 23, 2006 1:11 PM
To: pgsql-patches@postgresql.org
Subject: [PATCHES] Building with Visual C++

I've been working on getting the full backend to compile and run using
Visual C++ instead of mingw/gcc, and have made some good progress.
First, why do this? A couple of reasons:

1) MS VC++ is a significantly faster compiler, so rebuilding is much
faster ;)

2) MS VC++ supposedly has a better optimiser. I haven't got any pg
figures for this, but I've heard it from many other projects.

3) MS VC++ generates debug symbols in a way that's compatible with a
Windows debugger - such as the one in Visual Studio, but also the

basic

WinDbg debuggers that are the "standard windows debuggers". gdb is
horrible on win32.

4) It should also work with Visual Studio profiler tools. gprof is

also

pretty horrible on win32.

Unlike the previous stuff we do with VC++ this one doesn't use

separate

Makefiles. Instead I have a script that generates complete Visual

Studio

solution and project files by parsing the existing Makefiles. This

build

script isn't completed yet, there are a few too many things hardcoded

in

it. But it *works*, and the produced binary passes all regression

tests

except one (more on this in a separate mail in a minute or two). With
this it should also build fine with Visual C++ Express (free download
from MS), also not tested yet.

Turns out the changes are pretty small - most of the needed stuff was
implemented in /port already, and just needed activation for win32.

The attached patch makes some changes in the code required to work for
this, and some clenaups. A summary of the changes is:

*) WIN32_CLIENT_ONLY was a very bad name for a macro, since it pretty
much meant "visual c++". In a lot of places, it's just removed (need
removed by changing headers), in the few places where it's relevant we
instead use _MSC_VER which properly indicates Visual C++.

*) Change all open() commands to use the three parameter version. This
is because Visual C++ didn't support variadic macros until version 8

(VS

2005). It's only a very few places that used the two parameter

version,

so I think it's fine to make this change for all platforms.

*) Change where NaN is generated. Using 0.0/0.0 doesn't work in VC++
because it detects it as a divide-by-zero at the compile stage.

Storing

the second 0.0 in a variable makes this work. I made the change for

all

platforms to avoid #ifdef, but if that's not acceptable it can of

course

easily be made inside a #ifdef.

*) Cleanup incorrect definitions of several WINAPI functions. They

were

simply wrong, but mingw/gcc accepted it anyway..

*) Sync up pg_config.h.win32 to be a mingw-configure-output version

with

a couple of minor changes (listed). This is the largest part of the
patch...

*) Add s_lock implementation based on InterlockedCompareExchange

instead

of assembly code (gcc style asm not supported, and this should

probably

be about as fast from what I hear)

*) It also updates the old Visual C++ makefiles to still work. I

expect

them to go away once I'm done with the buildscript, but for now they
needed some small updates.

Finally, a couple of files are added to the tree:

src/include/port/win32vc/* - stub include files the same way
src/include/port/win32 is used already.

src/port/dirent.c - opendir/readdir/closedir implementation for win32,
that doesn't even have the bugs mingw had in it ;)

Hmm. That mail turned out longer than intended. Let's hope that means

I

Show quoted text

didn't forget anything...

//Magnus

I've done a similar thing myself (building the complete
backend with Visual Studio 2005). I got it to basically
work, but stopped due to lack of interest in the PostgreSQL community.

Well, I'm one of those previously with lack of interest :-)

The main difference between what I'm trying now from most of what I've
seen before is that I *don't* want to create a separate build
environment that needs to be maintained. Instead have a conversion
script from our current one to a VC compatible one.

But, I'd be happy to help out your effort in any way I can.

Thanks. There'll be plenty of pieces to pick up once I'm done with the
initial version of the script :-)

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

The main difference between what I'm trying now from most of what I've
seen before is that I *don't* want to create a separate build
environment that needs to be maintained. Instead have a conversion
script from our current one to a VC compatible one.

This is definitely the only way that the project will fly --- there's
no way we're going to duplicate our forest of Makefiles in a different
format that has to be hand-maintained. If Magnus can make a conversion
script work, though, I think we can accept that.

We'll need a VC buildfarm member in place to catch us anytime we change
the Makefiles in a way that the script doesn't understand. Before the
buildfarm existed I'd have been skeptical of whether even the
conversion-script approach would be viable in the long run, but with
timely info about breakages I think it can work.

BTW, has anyone looked at the possibility of driving VC from gmake,
so that we can continue to use the same Makefiles? Or is that just
out of the question?

regards, tom lane

BTW, has anyone looked at the possibility of driving VC from
gmake, so that we can continue to use the same Makefiles? Or
is that just out of the question?

I tried. It really didn't work. There are just too many assumption of
"unix cc style behaviour" in Make, and the VC compiler is just way too
different. If at all possible, I think we'd just end up bastardising our
Makefiles to a point where they'd be very hard to use on *any* platform.

Though to be complete, that was testing with autoconf as well. Didn't
get *anywhere* with autoconf, and didn't get very far at all with the
makefile parts.

//Magnus

The Microsoft VC++ compiler can definitely be driven by gmake (I've done
it at previous companies), but that involves a lot of platform-specific
knowledge added to the makefiles (.obj instead of .o, .exe instead of no
suffix, etc.).

Two downsides: One, is it makes the makefiles ugly, the other is that
it removes one benefit of native VC++ support: The ability to use
Visual Studio to build and debug in an integrated fashion.

Overall, I think an automated makefile to Visual Studio .vcproj
conversion can be made to work, and would be better.

-----Original Message-----
From: Tom Lane [mailto:tgl@sss.pgh.pa.us]
Sent: Monday, April 24, 2006 1:57 PM
To: Magnus Hagander
Cc: Chuck McDevitt; pgsql-patches@postgresql.org
Subject: Re: [PATCHES] Building with Visual C++

"Magnus Hagander" <mha@sollentuna.net> writes:

The main difference between what I'm trying now from most of what

I've

seen before is that I *don't* want to create a separate build
environment that needs to be maintained. Instead have a conversion
script from our current one to a VC compatible one.

This is definitely the only way that the project will fly --- there's
no way we're going to duplicate our forest of Makefiles in a different
format that has to be hand-maintained. If Magnus can make a

conversion

script work, though, I think we can accept that.

We'll need a VC buildfarm member in place to catch us anytime we

change

Show quoted text

the Makefiles in a way that the script doesn't understand. Before the
buildfarm existed I'd have been skeptical of whether even the
conversion-script approach would be viable in the long run, but with
timely info about breakages I think it can work.

BTW, has anyone looked at the possibility of driving VC from gmake,
so that we can continue to use the same Makefiles? Or is that just
out of the question?

regards, tom lane

The Microsoft VC++ compiler can definitely be driven by gmake
(I've done it at previous companies), but that involves a lot
of platform-specific knowledge added to the makefiles (.obj
instead of .o, .exe instead of no suffix, etc.).

It also requires major hackery to make all the commandline switches work
the proper way for different files etc. That's where the big difference
is.

Two downsides: One, is it makes the makefiles ugly, the
other is that it removes one benefit of native VC++ support:
The ability to use Visual Studio to build and debug in an
integrated fashion.

That's also a good point, and one which does make that part more
attractive yet.

Plus - compiling from a .vcproj can be orders of magnitude faster than
from a makefile (even win32 nmake-files) because the compiler is
launched in a much more efficient way)

//Magnus

-----Original Message-----
From: pgsql-patches-owner@postgresql.org
[mailto:pgsql-patches-owner@postgresql.org] On Behalf Of Tom Lane
Sent: 24 April 2006 21:57
To: Magnus Hagander
Cc: Chuck McDevitt; pgsql-patches@postgresql.org
Subject: Re: [PATCHES] Building with Visual C++

We'll need a VC buildfarm member in place to catch us anytime
we change the Makefiles in a way that the script doesn't
understand. Before the buildfarm existed I'd have been
skeptical of whether even the conversion-script approach
would be viable in the long run, but with timely info about
breakages I think it can work.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

Regards, Dave.

-----Original Message-----
From: pgsql-patches-owner@postgresql.org
[mailto:pgsql-patches-owner@postgresql.org] On Behalf Of Dave Page
Sent: 24 April 2006 22:35
To: Tom Lane; Magnus Hagander
Cc: Chuck McDevitt; pgsql-patches@postgresql.org
Subject: Re: [PATCHES] Building with Visual C++

-----Original Message-----
From: pgsql-patches-owner@postgresql.org
[mailto:pgsql-patches-owner@postgresql.org] On Behalf Of Tom Lane
Sent: 24 April 2006 21:57
To: Magnus Hagander
Cc: Chuck McDevitt; pgsql-patches@postgresql.org
Subject: Re: [PATCHES] Building with Visual C++

We'll need a VC buildfarm member in place to catch us anytime we
change the Makefiles in a way that the script doesn't understand.
Before the buildfarm existed I'd have been skeptical of

whether even

the conversion-script approach would be viable in the long run, but
with timely info about breakages I think it can work.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

Y'know I say that, but thinking about it I imagine the buildfarm is tied
well and truly into the GNU build system - running configure, make,
regression tests etc. We'd need to talk Andrew into modifying the code
to handle the different build procedure, and figure out how to run the
regression tests (prolly still need mingw for that - though possibly
SFU/Interix would work).

Regards, Dave.

"Dave Page" <dpage@vale-housing.co.uk> writes:

We'll need a VC buildfarm member in place to catch us anytime we
change the Makefiles in a way that the script doesn't understand.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

Y'know I say that, but thinking about it I imagine the buildfarm is tied
well and truly into the GNU build system - running configure, make,
regression tests etc. We'd need to talk Andrew into modifying the code
to handle the different build procedure, and figure out how to run the
regression tests (prolly still need mingw for that - though possibly
SFU/Interix would work).

Yeah, I had just started to wonder about that when your mail arrived.
It'll need to happen if we want to say that "VC is supported" with a
straight face, but I can easily imagine that it'll require nontrivial
work.

I don't mind though if running the regression tests requires mingw;
that's not an essential step unless you're a developer.

regards, tom lane

We'll need a VC buildfarm member in place to catch us anytime we
change the Makefiles in a way that the script doesn't understand.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

Y'know I say that, but thinking about it I imagine the buildfarm is
tied well and truly into the GNU build system - running configure,
make, regression tests etc. We'd need to talk Andrew into modifying
the code to handle the different build procedure, and

figure out how

to run the regression tests (prolly still need mingw for

that - though

possibly SFU/Interix would work).

Yeah, I had just started to wonder about that when your mail arrived.
It'll need to happen if we want to say that "VC is supported"
with a straight face, but I can easily imagine that it'll
require nontrivial work.

I don't mind though if running the regression tests requires
mingw; that's not an essential step unless you're a developer.

The tests I run against it now are run from MingW. And naturally, it
only works with "installcheck" and not "check", but that's to be
expected..

//Magnus

From: "Magnus Hagander"

I am sorry in a very late reaction.......

We'll need a VC buildfarm member in place to catch us anytime we
change the Makefiles in a way that the script doesn't understand.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

This is a trial in the reason I have not fully investigated your code yet.
And construction was tried by VC++6. It contains some problems.
I know that this differs from the solution which you consider.
However, I have some hope. As for present condition is the following situations.

1) Complain by access of a data directory.
C:\pgsql>bin\initdb -E EUC_JP --no-locale -Ddata -LC:/pgsql/share
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale C.
initdb: could not access directory "data": No error

2) $libdir is not looked for.
C:\pgsql>bin\initdb -E EUC_JP --no-locale -Ddata -LC:/pgsql/share
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale C.
fixing permissions on existing directory data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers/max_fsm_pages ... 4000/200000
creating configuration files ... ok
creating template1 database in data/base/1 ... ok
initializing pg_authid ... ok
enabling unlimited row size for system tables ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... FATAL: could not access file "$libdir/ascii_and_mic":
No such file or directory
child process was terminated by signal 1
initdb: removing contents of data directory "data"

3) could not select a suitable default timezone.
C:\pgsql>bin\postmaster -i -Ddata
FATAL: could not select a suitable default timezone
DETAIL: It appears that your GMT time zone uses leap seconds. PostgreSQL does
not support leap seconds.

---

However, it shows sufficient reaction. :-)
Although arrangement has not been carried out yet, a source code does not become
dirty so much.
This is a source code and a binary.(sorry, not diffs text.)
http://inetrt.skcapi.co.jp/~saito/pgsql82dev/

I also want this approach to be included in the following version.
Some of any suggestion?

Regards,
Hiroshi Saito

I am sorry in a very late reaction.......

No problem.

We'll need a VC buildfarm member in place to catch us

anytime we

change the Makefiles in a way that the script doesn't

understand.

Not a problem - Snake has VC++ 6, 2003 and 2005 installed.

This is a trial in the reason I have not fully investigated
your code yet.
And construction was tried by VC++6. It contains some problems.
I know that this differs from the solution which you consider.
However, I have some hope. As for present condition is the
following situations.

1) Complain by access of a data directory.
C:\pgsql>bin\initdb -E EUC_JP --no-locale -Ddata
-LC:/pgsql/share The files belonging to this database system
will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale C.
initdb: could not access directory "data": No error

I saw similar error when the code in dirent.c was broken. I'd look
there.

2) $libdir is not looked for.
C:\pgsql>bin\initdb -E EUC_JP --no-locale -Ddata
-LC:/pgsql/share The files belonging to this database system
will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale C.
fixing permissions on existing directory data ... ok creating
subdirectories ... ok selecting default max_connections ...
100 selecting default shared_buffers/max_fsm_pages ...
4000/200000 creating configuration files ... ok creating
template1 database in data/base/1 ... ok initializing
pg_authid ... ok enabling unlimited row size for system
tables ... ok initializing dependencies ... ok creating
system views ... ok loading system objects' descriptions ...
ok creating conversions ... FATAL: could not access file
"$libdir/ascii_and_mic":
No such file or directory
child process was terminated by signal 1
initdb: removing contents of data directory "data"

I think I saw that one with a broken pg_config_paths.h. (The one
generated by the vc++ makefiles in the current tree will generate an
invalid one for the backend (perfectly ok for frontend), the MingW build
system will generate a working one. My build-file-parsing-script fixes
it for vc++ as well.)

3) could not select a suitable default timezone.
C:\pgsql>bin\postmaster -i -Ddata
FATAL: could not select a suitable default timezone
DETAIL: It appears that your GMT time zone uses leap
seconds. PostgreSQL does not support leap seconds.

I saw this as well when the dirent.c stuff didn't work properly. It
could also be because of pg_config_paths.h.

//Magnus