Visual Studio 2010/Windows SDK 7.1 support

On Mon, Jan 3, 2011 at 08:19, Brar Piening <brar@gmx.de> wrote:

Hi,

i'v created a patch enables support for building PostgreSQL with Visual
Studio 2010 or Microsoft Windows SDK for Windows 7 and .NET Framework 4
(Windows SDK 7.1).
You can grab it from http://www.piening.info/VS2010.patch

This patch does not apply at all to my repository. Every single hunk
fails - I have a feeling it might have double line-ending encodings or
something like that? Since you're using git (at least eh patch
indicates so), have you perhaps published a git branch somewhere that
I could try pulling from instead?

Or if not - just to be sure, you are basing this off the master branch, I hope?

It's diffed against current head + running "perltidy -b -bl -nsfs -naws
-l=100 -ole=unix *.pl *.pm" as described in the README file (which seems not
to have been run before committing Mkvcbuild.pm the last time).

Yeah, it looks that way - it's missing the ordering of the contrib
arrays. I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

If you still see any changes on top of that, then there's something
strange going on..

It is problably neither the perfect way to introduce VS2010 support (my perl
is better than my C but probably still not what you are used to) nor is it
my way to try to make you officially support VS 2010. But perhaps it's
something you could start with once you decide to upgrade the msvc
toolchain.

Oh, starting with a patch is definitely not the wrong way :-)

it does look like they changed the file format extensively this time
though - that's annoying, but hopefully they will stick to the new
format this time...

The SDK 7.1 build produces tons of warnings which are mostly macro
redefinitions of EIDRM, EMSGSIZE, EAFNOSUPPORT, EWOULDBLOCK, ECONNRESET,
EINPROGRESS, ENOBUFS, EPROTONOSUPPORT, ECONNREFUSED and EOPNOTSUPP which
seem to have found their way into errno.h finally. Cutting those out of
src\include\pg_config_os.h and src\interfaces\libpq\win32.h makes the
project build pretty clean.
I resisted the temptation to parse them out of those files during
Mkvcbuild::mkvcbuild as this should probably be handled by some preprocessor
defines.

Yes, it's good that you resisted that :-)

It should be done with something like
#if _MSC_VER >= 1400

(with 1400 replaced with whatever version is appropriate)

There's at elast one example in c.h already, and several others spread
through files and code.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-------- Original-Nachricht --------

Datum: Mon, 3 Jan 2011 10:44:19 +0100
Von: Magnus Hagander <magnus@hagander.net>
An: Brar Piening <brar@gmx.de>
CC: pgsql-hackers@postgresql.org
Betreff: Re: [HACKERS] Visual Studio 2010/Windows SDK 7.1 support

This patch does not apply at all to my repository. Every single hunk
fails - I have a feeling it might have double line-ending encodings or
something like that?

double line-endings indeed!
Surprisingly only some of them (<CR><CR><LF> follow some <CR><LF>'s).

I was quite unsure about the best line endings for this patch so I probably messed it up by converting them forth and back. Still I'm not sure if there's something wrong with my git config - perhaps due to using git-external-diff (I wasn't able to find win32 binaries for filterdiff).

Sorry about that!

perl -e "open(INFILE, '<VS2010.patch');open(OUTFILE, '>VS2010_fixed.patch');binmode(INFILE);binmode(OUTFILE);$/=\"\015\015\012\";$\=\"\015\012\";while(<INFILE>){chomp; print OUTFILE;}close(INFILE);close(OUTFILE)"

fixes the problem.

Yeah, it looks that way - it's missing the ordering of the contrib
arrays. I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

If you still see any changes on top of that, then there's something
strange going on..

No that was probably all.

My patch - fixed as described above - should apply to your repository once you've run perltidy.

I'll rebase the patch as soon as I return from work unless you tell me otherwise.

Best Regards

Brar
--
GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
gratis Handy-Flat! http://portal.gmx.net/de/go/dsl

On 01/03/2011 06:26 AM, Brar Piening wrote:

No that was probably all.

My patch - fixed as described above - should apply to your repository once you've run perltidy.

I'll rebase the patch as soon as I return from work unless you tell me otherwise.

Please, this time let's backpatch the fixes to all supported branches
(from the buildfarm's POV that means back to 8.3 on for MSVC). I have
just been wrestling with the fact that we didn't do that with the VS
2008 changes, and these changes are much, much more extensive.

cheers

andrew

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander
<magnus@hagander.net> wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

It's actually the same patch as before with the corrupted line-endings
parsed out.

"git diff master VS2010 > ../VS2010.patch" reproduceabyl generates those
on my (windows) system when I use git-external-diff configured as
described in
http://wiki.postgresql.org/wiki/Working_with_Git#Context_diffs_with_Git
to produce context diffs.

Unified diffs (git diff master VS2010 --no-ext-diff >
../VS2010_unified.patch) only contain valid unix linfeeds

Perhaps it has got something to do with core.autocrlf=true but I don't
really care as long as I can get rid of it with one line of perl ;-)

For the records I'm using
$ git --version
git version 1.7.3.1.msysgit.0

and the configuration of the postgresql repository is
$ git config --list
core.symlinks=false
core.autocrlf=true
color.diff=auto
pack.packsizelimit=2g
help.format=html
http.sslcainfo=/bin/curl-ca-bundle.crt
sendemail.smtpserver=/bin/msmtp.exe
user.name=Brar Piening
user.email=[myemail]
core.repositoryformatversion=0
core.filemode=false
core.bare=false
core.logallrefupdates=true
core.symlinks=false
core.ignorecase=true
core.hidedotfiles=dotGitOnly
remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
remote.origin.url=git://git.postgresql.org/git/postgresql.git
branch.master.remote=origin
branch.master.merge=refs/heads/master
diff.external=git-external-diff

Best regards,

Brar

On Mon, Jan 3, 2011 at 16:45, Brar Piening <brar@gmx.de> wrote:

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander <magnus@hagander.net>
wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

Thanks. It still has windows linebreaks, but only once now, so they
can be properly removed by patch. It applies cleanly now.

It's actually the same patch as before with the corrupted line-endings
parsed out.

"git diff master VS2010 > ../VS2010.patch" reproduceabyl generates those on
my (windows) system when I use git-external-diff configured as described in
http://wiki.postgresql.org/wiki/Working_with_Git#Context_diffs_with_Git to
produce context diffs.

Yeah, I'm not sure those instructions really take Windows into account :-)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 01/05/2011 12:33 PM, Magnus Hagander wrote:

It's actually the same patch as before with the corrupted line-endings
parsed out.

"git diff master VS2010> ../VS2010.patch" reproduceabyl generates those on
my (windows) system when I use git-external-diff configured as described in
http://wiki.postgresql.org/wiki/Working_with_Git#Context_diffs_with_Git to
produce context diffs.

Yeah, I'm not sure those instructions really take Windows into account :-)

I've abandoned the use of an external diff program for git. If I need
context diffs I produce them by piping normal git diff output through
"filterdiff --format=context".

cheers

andrew

On Wed, Jan 5, 2011 at 12:59 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

I've abandoned the use of an external diff program for git. If I need
context diffs I produce them by piping normal git diff output through
"filterdiff --format=context".

One possible disadvantage of that it strips out this stuff:

diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 6a9b21d..1c6ae02 100644

I actually haven't figured out exactly under what circumstances that
index information is used by git, but I think in theory it's supposed
to make it possible for git to apply patches that would otherwise
fail. However, I actually haven't figured out the best way to make
git do that, either; I'm still using patch -p1, which is basically
horrible if there are >0 conflicts.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:

diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 6a9b21d..1c6ae02 100644

I actually haven't figured out exactly under what circumstances that

I think it's for git am -3

http://www.kernel.org/pub/software/scm/git/docs/git-am.html

But I don't see how to use that on a patch directly, the UI seems to
expect a mail file (mbox or Maildir formats).

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support

On Mon, Jan 3, 2011 at 16:45, Brar Piening <brar@gmx.de> wrote:

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander <magnus@hagander.net>
wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

Hi!

Please make sure this goes on the commitfest page
(https://commitfest.postgresql.org/action/commitfest_view?id=9), so
it's not missed.

I'm currently lacking an env where I can put VS2010 on it (given that
the amazon cloud no longer works reasonably for windows machines, and
I can't put another version of VS on the other VM I'm using right
now), so it'll be a while before I can look at this.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 1/15/2011 1:42 PM, Magnus Hagander wrote:

On Mon, Jan 3, 2011 at 16:45, Brar Piening<brar@gmx.de> wrote:

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander<magnus@hagander.net>
wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

Hi!

Please make sure this goes on the commitfest page
(https://commitfest.postgresql.org/action/commitfest_view?id=9), so
it's not missed.

I'm currently lacking an env where I can put VS2010 on it (given that
the amazon cloud no longer works reasonably for windows machines, and
I can't put another version of VS on the other VM I'm using right
now), so it'll be a while before I can look at this.

I can provide a windows box with VS2010 if you'd like. Wouldn't be
until Monday or Tuesday. Any preference on windows version? Maybe
Windows 7? You want 64-bit?

Send a private email.

--
Andrew Chernow
eSilo, LLC
global backup
http://www.esilo.com/

Hi,

I initially considered this patch as a primer to start off some basic
VS2010 support and not as something to be commited within the next few
commitfests.

As there seems to be at least some interest in this patch I refactored
the code and did some more testing (actually found some weird issues
with pgbison.bat and pgflex.bat and replaced them with perl variants).

So there is now a third version of this patch at
http://www.piening.info/VS2010v3.patch

This time even with unix linefeeds ;-)

On Sat, 15 Jan 2011 19:42:06 +0100, Magnus Hagander
<magnus@hagander.net> wrote:

Please make sure this goes on the commitfest page
(https://commitfest.postgresql.org/action/commitfest_view?id=9), so
it's not missed.

I'll add it.

Regards,

Brar

On Tue, 18 Jan 2011 00:06:10 +0100, Brar Piening <brar@gmx.de> wrote:

So there is now a third version of this patch at
http://www.piening.info/VS2010v3.patch

Forgot to run perltidy on it - fixed in
http://www.piening.info/VS2010v4.patch

Sorry!

Brar

On Sat, Jan 15, 2011 at 1:42 PM, Magnus Hagander <magnus@hagander.net> wrote:

On Mon, Jan 3, 2011 at 16:45, Brar Piening <brar@gmx.de> wrote:

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander <magnus@hagander.net>
wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

Hi!

Please make sure this goes on the commitfest page
(https://commitfest.postgresql.org/action/commitfest_view?id=9), so
it's not missed.

I'm currently lacking an env where I can put VS2010 on it (given that
the amazon cloud no longer works reasonably for windows machines, and
I can't put another version of VS on the other VM I'm using right
now), so it'll be a while before I can look at this.

Magnus, are you planning to get this committed for 9.1?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Sun, Jan 30, 2011 at 21:06, Robert Haas <robertmhaas@gmail.com> wrote:

On Sat, Jan 15, 2011 at 1:42 PM, Magnus Hagander <magnus@hagander.net> wrote:

On Mon, Jan 3, 2011 at 16:45, Brar Piening <brar@gmx.de> wrote:

On Mon, 3 Jan 2011 10:44:19 +0100, Magnus Hagander <magnus@hagander.net>
wrote:

Yeah, it looks that way - it's missing the ordering of the contrib
I'll run it once for that now, and then please rebase your
patch on top of that - makes it easier to review it.

The rebased patch can be grabbed from http://www.piening.info/VS2010v2.patch

Hi!

Please make sure this goes on the commitfest page
(https://commitfest.postgresql.org/action/commitfest_view?id=9), so
it's not missed.

I'm currently lacking an env where I can put VS2010 on it (given that
the amazon cloud no longer works reasonably for windows machines, and
I can't put another version of VS on the other VM I'm using right
now), so it'll be a while before I can look at this.

Magnus, are you planning to get this committed for 9.1?

I'd like to, but I'm not sure I'll have the time. Per the comment from
Brar it was intended as an initial attempt not quite ready yet, so
it's not something we should hold up the CF / release for.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Sun, Jan 30, 2011 at 3:26 PM, Magnus Hagander <magnus@hagander.net> wrote:

Magnus, are you planning to get this committed for 9.1?

I'd like to, but I'm not sure I'll have the time. Per the comment from
Brar it was intended as an initial attempt not quite ready yet, so
it's not something we should hold up the CF / release for.

OK, I'm marking it Returned with Feedback. We can resurrect it if need be.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Sun, 30 Jan 2011 21:26:22 +0100, Magnus Hagander
<magnus@hagander.net> wrote:

it's not something we should hold up the CF / release for.

I agree.
At least it should get some more testing besides mine.

I've set up virtual machines with VS 2003, VS 2005 Express, VS 2008
Express (+ my PC with VS 2010) for testing purposes but I didn't test
all possible build paths with respect to the external libraries to include.
While I didn't change much of the existing VS 2005/8 code I currently
can't guarantee that the VS 2010 build will work for every possible
external library one could include (yet I didn't stumble into any
failure while testing) and still I could have broken some VS 2005/8
build path too.
The patch could also be extended to automatically support building libpq
when VS 2003 is detected or support other desireable features that
aren't really in the context of supporting VS 2010.

Being somewhat short of time in the next weeks I'm at least willing to
rebase the patch on request and do some more testing or fix issues
someone else has detected before the next release (9.2?) goes beta.

If there's some pressure to support VS 2010 asap - please let me know
and I'll see what I can do.

Best regards,

Brar

On Sun, Feb 6, 2011 at 23:06, Brar Piening <brar@gmx.de> wrote:

On Sun, 30 Jan 2011 21:26:22 +0100, Magnus Hagander <magnus@hagander.net>
wrote:

it's not something we should hold up the CF / release for.

I agree.
At least it should get some more testing besides mine.

I've set up virtual machines with VS 2003, VS 2005 Express, VS 2008 Express
(+ my PC with VS 2010) for testing purposes but I didn't test all possible
build paths with respect to the external libraries to include.

Yeah, the external libraries are really the biggest thing.

While I didn't change much of the existing VS 2005/8 code I currently can't
guarantee that the VS 2010 build will work for every possible external
library one could include (yet I didn't stumble into any failure while
testing) and still I could have broken some VS 2005/8 build path too.
The patch could also be extended to automatically support building libpq
when VS 2003 is detected or support other desireable features that aren't
really in the context of supporting VS 2010.

Being somewhat short of time in the next weeks I'm at least willing to
rebase the patch on request and do some more testing or fix issues someone
else has detected before the next release (9.2?) goes beta.

Sounds good.

If there's some pressure to support VS 2010 asap - please let me know and
I'll see what I can do.

I don't think there is, really. It's a "nice to have", but if it comes
in 9.2 instead of 9.1, I don't think that's a problem. 99.9% of all
Win32 users don't build from source in the first place, and I'm sure
Dave is happy not to have to dela with another version ;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Sun, 06 Feb 2011 23:06:21 +0100, Brar Piening <brar@gmx.de> wrote:

On Sun, 30 Jan 2011 21:26:22 +0100, Magnus Hagander
<magnus@hagander.net> wrote:

it's not something we should hold up the CF / release for.

I agree.
At least it should get some more testing besides mine.

[...]

Being somewhat short of time in the next weeks I'm at least willing to
rebase the patch on request and do some more testing or fix issues
someone else has detected before the next release (9.2?) goes beta.

After some months of being able to regularly compile current head using
Visual Studio 2010 compilers and some more tests I no longer see any
reason why this patch would change without external feedback.

I've tested the latest version (http://www.piening.info/VS2010v6.patch)
with the following config.pl for x86

$config->{perl} = 'C:\Perl'; # ActivePerl 5.8.9 Build 829
$config->{tcl} = 'C:\Tcl'; # ActiveState ActiveTcl 8.4.19.5
$config->{python} = 'C:\Python27'; # Python 2.7.1
$config->{openssl} = 'C:\openssl'; # openssl-1.0.0d
$config->{nls} = 'C:\Dev\gnuwin32'; # GetText 0.14.4
$config->{krb5} = 'C:\Dev\kfw-3-2-2-final';
$config->{xml} = 'C:\Dev\libxml2-2.7.7.win32';
$config->{xslt} = 'C:\Dev\libxslt-1.1.26.win32';
$config->{iconv} = 'C:\Dev\iconv-1.9.2.win32';
$config->{zlib} = 'C:\Dev\zlib125';

for x64 I've tested a less extensive configuration as it's still hard to
get the above libraries as x64 binaries.

I'd appreciate any reviews, tests or comments.

If someone with some more C skills than me could silence the warnings
this would be a huge gain for build speed and testing. My previous
attempts silenced the warnings but built a non-connectable backend.

Regards,

Brar

On Fri, May 13, 2011 at 5:34 PM, Brar Piening <brar@gmx.de> wrote:

After some months of being able to regularly compile current head using
Visual Studio 2010 compilers and some more tests I no longer see any reason
why this patch would change without external feedback.

You probably want to add it here, then:

https://commitfest.postgresql.org/action/commitfest_view/open

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company