Re: [pgsql-hackers-win32] Build with Visual Studio & MSVC

On 5/4/06, Gurjeet Singh <singh.gurjeet@gmail.com> wrote:

My main grudge is that if we are supporting almost all flovours of
nixens and compilers (close to 34 according to official website), then
why are we leaving Windows platform alone? This will bring in quite a
lot more developers.

Sorry, but this sounds pretty rude. If you have a grudge, do
something about it and stop whining about our support for tons of *nix
platforms instead of Windows. For the most part, the reason *nix
platform support is more popular is because that's what most of us use
on a daily basis.

Keep in mind this is an open source project and a lot of us have more
important things to work on than simple porting. It's easy to
criticize, it's much harder to do the actual work.

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

Gurjeet Singh wrote:

My main grudge is that if we are supporting almost all flovours of
nixens and compilers (close to 34 according to official website), then
why are we leaving Windows platform alone? This will bring in quite a
lot more developers.

You should look at MinGW as a development toolkit, not a platform. PostgreSQL builds and
runs just fine on the Windows platform. Personally, I use Eclipse C/C++ with MinGW since it
brings me a number of advantages. The most prominent one is that I only need to master one
IDE regardless of platform.

I am sure it's not going to be easy, but I am sure with this great
community suppport, we sure can achieve it.

Seems some people has done a lot of work to get things working with VC++ already. Search for
the word MSVC on this list.

Regards,
Thomas Hallgren

Hi William(uniware), Chuck and Hackers,

I have been interested in doing complete PGSQL
development in MSVC for a long time now. With reference to
one of Chuck's mails to
-hackers-win32 with the same subject, you said that you were
able to successfully compile PG 8.1 with some minor tweaks.

Also, William has 'vcproject' hosted on pgfoundry, I
downloaded it, and tried compiling
vcproject\msvc\postgres\postgres.dsw on
VC++6.0. It failed miserably with over 1000 errors. I am sure there's
some tweaks needed here too!!!

Yes. There is a patch pending on -patches which fix almost all of these
in HEAD. (There are a few tiny things related to perl and NLS that
aren't included in it ATM. And I'm just assuming you're seeing the same
problems as I was but I didn't base my work off vcproject). I'm also
working on a buildscript to convert the Makefiles to visual c++ project
files, but that's not quite done yet. The idea with this work is to have
the stuff as integrated as possible with main CVS, so the maintenance
will be as low as possible - unlike the vcproject project which has been
focusing on keeping a separate build environment maintained.

The target is VC++ 2003 and 2005 ATM, but it should just be a matter of
a different output format for VC 6.0 I guess.

You will still need things like bison and flex if you want to build off
cvs, of course - there is no builtin support for that in VC++.

//Magnus

Yes. There is a patch pending on -patches which fix almost all of
these in HEAD. (There are a few tiny things related to perl and NLS
that aren't included in it ATM. And I'm just assuming you're seeing
the same problems as I was but I didn't base my work off

vcproject).

I'm also working on a buildscript to convert the Makefiles

to visual

c++ project files, but that's not quite done yet. The idea

with this

work is to have the stuff as integrated as possible with

main CVS, so

the maintenance will be as low as possible - unlike the vcproject
project which has been focusing on keeping a separate build

environment maintained.

The CrystalSpace and PlaneShift projects I'm associated with
have automated MSVC generators. Want me to try to grab them
for you? They'd be GPL though.

You mean they have a tool that parses GNU Makefiles and generate VC
project files? Sure, that might be interesting. I've seen I think two
others, and tried, but they fell over badly because the pg build system
was too complicated. But I beleive I'm still allowed to loko at GPL
stuff and get ideas as long as I don't copy the code :-)

//MAgnus

Yes. There is a patch pending on -patches which fix almost all of these
in HEAD. (There are a few tiny things related to perl and NLS that
aren't included in it ATM. And I'm just assuming you're seeing the same
problems as I was but I didn't base my work off vcproject). I'm also
working on a buildscript to convert the Makefiles to visual c++ project
files, but that's not quite done yet. The idea with this work is to have
the stuff as integrated as possible with main CVS, so the maintenance
will be as low as possible - unlike the vcproject project which has been
focusing on keeping a separate build environment maintained.

The CrystalSpace and PlaneShift projects I'm associated with have
automated MSVC generators. Want me to try to grab them for you? They'd
be GPL though.

Chris

You mean they have a tool that parses GNU Makefiles and generate VC
project files? Sure, that might be interesting. I've seen I think two
others, and tried, but they fell over badly because the pg build system
was too complicated. But I beleive I'm still allowed to loko at GPL
stuff and get ideas as long as I don't copy the code :-)

http://cvs.sourceforge.net/viewcvs.py/crystal/CS/mk/msvcgen/

I think it actually takes some sort of XML format or something...or just
reads in files in directories. I'm not sure.

Chris

----- Original Message -----
From: "Magnus Hagander"

The target is VC++ 2003 and 2005 ATM, but it should just be a matter of
a different output format for VC 6.0 I guess.

You will still need things like bison and flex if you want to build off
cvs, of course - there is no builtin support for that in VC++.

Ahh, Your right... However, I wish to offer construction of the DSP project of VC6.
I continues after you. But, This is not fully considered carefully yet.
http://inetrt.skcapi.co.jp/~saito/pgsql82dev/
Probably , Flex and bison do not need for the formal release souce of PostgreSQL.
It will already be generated.

Regards,
Horoshi Saito

`vcproject` is based on pgsql-8.0.3. It's purpose is to make pgsql built
with MSVC++.
But I found there are few people intrested on it, so I stopped maintaining
it months
ago. `vcproject` still need MSYS/MinGW, the basic idea behind it is:

1) Let we do configure, make, make install in MinGW first.
This step can make sure our source code is OK under MinGW, building with
GNU toolchain.
2) My major work is maintaining MSVC++'s special project files, including
*.dsp and *.dsw.
I have also do some minor changes to the source files, you can diff with
pgsql-8.0.3.
3) Finally, I can build the while system with MSVC++'s IDE or command line
MSVC.exe.

See README for details. The link is:
http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/vcproject/vcproject/README?rev=1
.8&content-type=text/x-cvsweb-markup

`vcproject` is not perfect, but it works for me. And I think it can work
with pgsql-8.1.

Sorry for the late response.

Regards,
William ZHANG

On Fri, May 05, 2006 at 09:50:38AM +0200, Magnus Hagander wrote:

You mean they have a tool that parses GNU Makefiles and generate VC
project files? Sure, that might be interesting. I've seen I think two
others, and tried, but they fell over badly because the pg build system
was too complicated. But I beleive I'm still allowed to loko at GPL
stuff and get ideas as long as I don't copy the code :-)

[Note: I have no idea how much people have done on this already. It's
just that all this talk of automatic generation makes me curious.
Myself, I have no idea how VC makefile work.]

Is it so hard to automatically generate the necessary info? On a clean
source tree, "make -n" will dump all the commands required to complete
the build. You could probably extract all the info you required from
there, although the directory changing would kill you.

So my thought is, create a number of tracing scripts, eg cc-trace which
examine their arguments to see what needs to be done, recording the
current directory and such. Then execute:

make CC="cc-trace" LD="ld-trace" etc...

And you should be able to build up a tree of what depends on what. This
doesn't take care of the other stuff the makefile does though (like the
generation of pg_config_paths.h, can VC makefile do things like that?)

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/

You mean they have a tool that parses GNU Makefiles and generate VC
project files? Sure, that might be interesting. I've seen I

think two

others, and tried, but they fell over badly because the pg build
system was too complicated. But I beleive I'm still allowed

to loko at

GPL stuff and get ideas as long as I don't copy the code :-)

[Note: I have no idea how much people have done on this
already. It's just that all this talk of automatic generation
makes me curious.
Myself, I have no idea how VC makefile work.]

Is it so hard to automatically generate the necessary info?

No. Not really :-)

On a clean source tree, "make -n" will dump all the commands
required to complete the build. You could probably extract
all the info you required from there, although the directory
changing would kill you.

Can't do that on a clean source tree, you need to ./configure first.

So my thought is, create a number of tracing scripts, eg
cc-trace which examine their arguments to see what needs to
be done, recording the current directory and such. Then execute:

make CC="cc-trace" LD="ld-trace" etc...

And you should be able to build up a tree of what depends on
what. This doesn't take care of the other stuff the makefile
does though (like the generation of pg_config_paths.h, can VC
makefile do things like that?)

Again, this requires msys and GNU make to start with. I want it to work
without those completely. And the script I have now does this just fine,
it just needs some cleanup work and the removing of a couple of
hardcoded things.
A lot of space in it right now is because I list each project manually
in the beginning (postgres, one for each binary in bin/ etc) because the
makefiles aren't entirely consistent for that. And it also gorws a bit
because the conversion procs are built a different way...

//Magnus

VC++6.0 isn't a very good compiler and it's not very compatible with
gcc, while Visual Studio 2005 compiler is much more
compatible and has a better optimizer.

Plus, VC++6.0 had a closed "proprietary" data format for .dsp
and .dsw files, while the current Visual Studio uses a
standard XML format.

Finally, Microsoft gives away (as in free, no cost) Visual
C++ Express edition, which includes the current compiler.

I don't see any reason we'd want to target VC++6.0.

P.s. With the current Visual Studio, it's easy to add Bison
and Flex custom rules, so that it automatically calls them
for .y and .l files.

Right. It can be done. It's not quite that easy though - we need to
rename the output files as well, the default names generated by
bison/flex won't work.

//Magnus

(missed the mailing list last time)
Hi William,

Thanks for the steps to succeed with vcproject. I am sure that
there will be a few more tweaks involved (like setting VC's include
path etc.). I will try to build it and consult/inform you if there is
any deviation from the README.

This will work as a starting point in understanding the
requirements for what I intend to do. As Magnus mentioned earlier, I
also wish to create the project that will not include the source files
like 'vcproject' does; instead, it will contain just the dsp and the
dsw (VC 6) files. Somebody interested in working with newer versions
of VS can use it's feature to upgrade these formats to the respective
version.

Chris, the on initial cursory look, the msvcgen.sh file stored in
the attic shows that your project uses some tool called jam to do the
magic. I would really love to have this functionality in the new port
we are developing. I will look deeper into it.

Thomas, I love the idea of eclipse; any platform, any language,
one IDE. I am downloading it right now. Can you please send in the
steps that you perform to setup the environment, including mingw
toolkit; I will try to grow on that. The idea of this effort is to
have a GUI IDE, with a slew of features that MSVC offers: Memory
window that allows you to edit memory inplace, call-stack window,
watches, quick-expression evaluater with class/struct support, etc.
etc. . If Eclipse can offer all these, then I dont think anyone would
mind using it insead of MSVC.

<Jonah>
...If you have a grudge, do something about it...
...a lot of us have more important things to work on than simple porting...
</Jonah>

That's exactly the point Mr Jonah. I am trying to do something,
and trying to leverage from the past experiences of others in the
community. Looking at these handful of the responses, it seems that I
am not alone in the quest for using a better IDE!!!

And, no one is forced to respond to a mail to criticize, rather
than help. If it is such a 'simple porting', may I ask why hasn't it
been attempted successfuly in so many years of PG's history? At least
I dont see anybody documening any such procedure. This 'simple
porting' will allow and pull-in a lot more talented devs than you and
I. Wish me luck in this effort Mr Jonah.

Gurjeet.

On 5/5/06, Jonah H. Harris <jonah.harris@gmail.com> wrote:

On 5/5/06, Gurjeet Singh <singh.gurjeet@gmail.com> wrote:

If it is such a 'simple porting', may I ask why hasn't it been
attempted successfuly in so many years of PG's history?

Because most of use don't use Windows... I thought I said that.

But a sizable number of pgsql's users (developers and others) do!!!

This 'simple porting' will allow and pull-in a lot more talented
devs than you and I.

Sorry, but when it comes to Windows, the word talent is extremely relative.

So is the case with Nixens too! For eg., nixen developers are
known for great programming talent, but when it comes to being
user-friendly and look-and-feel their creations suck! I dont know the
case now, but till a year ago (before it went open), even after 20
years in production, SUN Solaris' GUI was a shame.

I favour Windows over nixens, not because I like Billy, or because
Unix is Evil, but because, Windows has snatched the monopoly of
computers from geeks and brought it to the common man. It has enabled
the not-so-rich to buy a computer and hop on to the tech bandwagon;
which ultimately means more-to-do (jobs) for geeks!

It might be buggy, it might be slow, they might have stolen the
idea from Mac... I don't care; I am using one right now to communicate
with the world.

Let other's make fun of it... I respect it; and will continue to,
for the rest of my life, even if I ever start using Linux in future.

Wish me luck in this effort

Good luck!

Thank you Mr Jonah, I think your wish materialized and I have found a solution!

--
Jonah H. Harris, Database Internals Architect
EnterpriseDB Corporation
732.331.1324

Hi Everybody,

Finally I have succeeded in being able to debug pgsql using a GUI
IDE, and no, it is not MSVC; its Eclipse.

As a Windows developer alI I was looking for was a
simple-yet-effective GUI, that is capable of showing me multiple
facets of the project very easily, and allow me to switch settings
easly (radio buttons, check-boxes) instead of messing around with
cryptic makefiles and config files.

CDT (Eclipse plugin for C/C++) comes very close to what I (a
WinDev) wanted to get started with PG/OSS. It has nice configurable
GUI, breakpoints, watches, keyboard shortcuts, etc etc, almost
everything that I wanted. Best of all, the shortcuts are configurable;
although I think the defaults are for emacs users, but I can change
them to work like in VC. I say 'almost' because it is a
work-in-progress, and there are a few (major) performance bottlenecks
(especially the symbol-indexer).

It works seamlessly with the msys (mingw) toolkit. It parses the
make-output and builds it's project; no need to scratch your head and
which files are to be imported in the project. Then, it uses the gdb
in the background, to let you attach to an already-running process,
and lets you debug it, just lilke in VC. It surely is a dream come
true.

I spent few hours installing/uninstalling the msys toolkit, before
I got it right with the help from mingw's wiki pages. There were other
gotchas too after the toolkit started working; creating user postgres
and running postmaster under it, --with-debug flag to ./configure for
gdb work, hacking config file to disable gcc optimizations, having to
run Eclipse as user 'postgres', and many more....

The best part about it is that there is no need for any new
project to be created and maintained (like vcproject), nor are any
changes/inclusions required to the existing source-tree. Simply
checkout a CVS module, and start working with it!!!! On top of that,
it uses the pg-recommended toolkit for Windows; so, you can even
package and distribute your builds (not recommended though)!

Finally I have got what I wanted. Now I think I (and any other
WinDev) can work/learn/contribute to the PG and other OSS! I will put
together a complete, step-by-step procedure of how to set it all up,
so that one can get up and running in no time at all.

I am not going to pursue the VC or VS integration anymore, and I
highly recommend everyone to use this solution if you want to develop
PG/OSS on windows.

A big thank you to all,
Gurjeet.

PS: Here's a bit long PS where I took the liberty to pour my heart out:

I know that I (re-)started this thread with a request for MSVC
support. I like VC++6 very much; convenience of shortcuts for
everything (I hardly use mouse when working on VC), and the nice
configurable layout. I like it more than it's successors, VS.Net
versions (mouse is almost indispensable), and thats why I was pushing
for VC rather than VS.

But I am giving up on the idea because the situation demands it.

Last year I had tried my hands on building version 7.4 of PG using
the cygwin, but was put-off by gdb's kludgy interface when it came to
debugging; so I gave up on it and got busy with other things. Although
I have worked on Solaris and Linux for 1 and 1/2 years, and used
gcc-gdb for everything, but I couldn't get to terms with the fact that
such a great talent-pool of Unix/Linux developers were not able to pu
together a nice GUI in so many years.

Probably the fact that, in the first two years as a developer I
used VC 6, also kept pushing me away from the command-line-debugging.
Without knowing it, I fell in love with VC and I realized it only when
I had to start using gdb; I longed to get back to VC. I think unix
also played a big role in my leaving one of the dream-jobs at one of
the dream-companies.

I did get away from gdb, but couldn't get back to VC. I got VS.Net
in exchange, which looks like a more good-looking but high-maintenance
step-sister of a plain, simple and beautiful-in-its-own-way VC.

I am spending my office hours with VS, but I still keep VC on my
home machine. You never forget your first love...

Gurjeet Singh wrote:

Thomas, I love the idea of eclipse; any platform, any language,
one IDE. I am downloading it right now. Can you please send in the
steps that you perform to setup the environment, including mingw
toolkit; I will try to grow on that.

My setup is pretty basic. I can't give you an exact step by step instruction since it's been
a while since I last did it. I installed msys, mingw, Eclipse. and then, using the Eclipse
update manager and the 'Callisto' site, I installed the C/C++ plugin.

I also made sure that the PATH in effect for the Eclipse IDE contains entries for the
%MSYS_HOME%\bin and %MINGW_HOME%\bin. That's it basically.

The idea of this effort is to
have a GUI IDE, with a slew of features that MSVC offers: Memory
window that allows you to edit memory inplace, call-stack window,
watches, quick-expression evaluater with class/struct support, etc.
etc. . If Eclipse can offer all these, then I dont think anyone would
mind using it insead of MSVC.

Eclipse won't offer all of these. Not yet anyway. What you get is a fair C/C++ editor and
parsers for your make output that will annotate your files with errors and warnings. There's
said to be some debugging support too on top of gdb, but to be honest, I've never tried it
on Windows. I do my C-debugging using gdb on Linux. My attempts to use gdb on Windows have
been quite futile so far. Then again, I'm not using the latest MinGW version so perhaps
there's still hope.

All in all, Eclipse C/C++ has some way to go before it can match up with MSVC. My point was
that you can do Windows development without MSVC and you can do it fairly well. If you are a
Linux hacker, you might even prefer doing it that way. So as a platform, Windows is not by
any means "left alone".

I really think that what you and others are trying to accomplish is very valuable. If not
for me (since I'm mixing Java and C and work on multiple platforms) then certainly for many
others. Personally, I'd rather see a Visual Studio port than one for VC++6.0. I wish you the
best of luck.

Regards,
Thomas Hallgren

Hi All,

I know it has been over a week now since I said that I'll put
together a document that describes how to debug postgres using Eclipse
IDE on Windows. I have finally completed the first draft and uploaded
it here:

http://www.geocities.com/gurjeet79/pg_on_eclipse.txt

And here's a screenshot of the fruits it'd bear if someone tries
to follow it and go through with it:

http://www.geocities.com/gurjeet79/pg_on_eclipse.jpg

Hope it helps to get more people more familiar with the postgres code.

Thomas, thanks a lot for the background. IMO, now Eclipse + CDT is
very close to MSVC / VS.Net in terms of the features that I was
looking for. The debugger is well integrated now, and I haven't had
any issues in this last one week.

Since you said that you haven't tried debugging on Windows, I'd
request you to go through this document and see if you like what you
finally can do within Eclipse. Any inputs on the document will be
appreciated.

Thanks,
Gurjeet.

Gurjeet,

    I know it has been over a week now since I said that I'll put
together a document that describes how to debug postgres using Eclipse
IDE on Windows. I have finally completed the first draft and uploaded
it here:

Hey, are you a member of the Eclipse Project? Mike was bugging me a few
months ago about wanting more involvement by PostgreSQL people in their Data
Tools project.

--
Josh Berkus
Aglio Database Solutions
San Francisco

No I am not... I used Eclipse for the first time just last week.
But yes, I wish to contribute to the CDT plugin. I think their Indexer
is a bit slow... it takes more than an hour (about two hours) to index
postgres' source code!!! Also, I just noticed that the background gdb
crashes when trying to open an uninitialized structure in the
'Variables' view.

If you are talking about, Mike Taylor from Eclipse,
(http://www.eclipse.org/org/elections/candidate.php?year=2006&amp;id=mtaylor),
then I'd be delighted to get in touch with him and see if Postgres and
Eclipse can work more closely.

Gurjeet.

Josh,
I'm the lead architect of the Eclipse Buckminster project
(www.eclipse.org/buckminster). I'd be happy to help the guys from data
tools with PostgreSQL if there's anything I can do. Not sure what that
would be though. Which Mike is it you're referring to?

Regards,
Thomas Hallgren

Gurjeet Singh wrote:

I just recently came to know that geocities.com is down. It is
accessible from geocities.yahoo.com though, but my site is still not
accessible. I had deleted my local copy, relying on the gecities, and
almost lost it; but then I twisted some knobs in geocities.yahoo.com
to get the file back...

Since I almost lost it, I thought it'd be a good idea to mail it
across to everyone this time, so that there is at least one source
when needed....!!!

Regards,
Gurjeet.