Building Windows Server Extensions Using VC++ 2005

Charles F. I. Savage wrote:

Hi everyone,

I've been able to successfully build server extension using Visual
Studio 2005 for Windows Postgresql 8.1. However, it took a few tweaks
which I thought I should document (maybe these issues could be fixed in
future postgresql versions?):

Sorry for the delay in replying to your issues.

1. There is no lib file for VC++ to link against. This can be created
easily enough by going to src/backend directory and running:

lib /MACHINE:x86 /NAME:postgres.exe /DEF:postgres.def

Note the use of /NAME to tell VC++ it is linking against an executable
and not a dll.

It would be nice if this lib file was automatically installed on windows
when installing postgresql.

We don't distribute a *.def file in this case because it would be too
hard to keep it up-to-date. We have *.def files for libpq, but that
changes much less frequently than the backend. The most reliable
solution is for users to generate the file themselves.

2. Requirement on strings.h

In c.h:69 there is this code:

#ifdef HAVE_STRINGS_H
#include <strings.h>
#endif

In pg_config.h:405 this is defined:

/* Define to 1 if you have the <strings.h> header file. */
#define HAVE_STRINGS_H 1

However, Visual Studio 2005 does not include this file. For a
workaround I simply added it but that's a bit of hack.

Ah, so even though you are using MSVC, you are using the config file
from MinGW. Easy fix, included in the patch below, and applied for 8.2.

3. This is a bigger issue, and took a while to figure out. If you try
to use the Version 1 calling convention, your function will be called
but if you try to do anything with the passed in parameters a
segmentation fault will occur. If you use the Version 0 calling
convention things work fine.

The problem is if you use PG_FUNCTION_INFO_V1 postgres does not see the
generated function because it is not exported from the dll and thus
assumes the Version 0 calling convention when in fact your function is
using Version1. The problem is in fmgr.h:298

#define PG_FUNCTION_INFO_V1(funcname) \

extern Pg_finfo_record * CppConcat(pg_finfo_,funcname) (void); \
...

For windows to export this function it must be:

extern __declspec(dllexport) Pg_finfo_record *
CppConcat(pg_finfo_,funcname) (void);

Would it be possible to add a DLLEXPORT macro here to fix this?

Yes, we can add DLLIMPORT (which is our macro for that).

However, looking at why /contrib works for MinGW, I think it is the use
of this from Makefile.shlib:

$(DLLTOOL) --export-all $(DLLTOOL_DEFFLAGS) --output-def $(NAME).def $(OBJS)

I am thinking you should just do the same for MSVC. Should we add
DLLIMPORT to PG_FUNCTION_INFO_V1()? It duplicates --export-all but it
seems like the correct solution.

4. Last, and the biggest issue, if my function calls pfree it blows
up. What is happening is that pfree somehow ends up pointing to a
random spot in memory - thus when you try to call it you execute invalid
code (in fact you never execute pfree at all as far as I can see). I
worked around this by using pgport_pfree which does work. Haven't a
clue why...

The answer is in a comment just above pgport_palloc():

/*
* Memory support routines for libpgport on Win32
*
* Win32 can't load a library that DLLIMPORTs a variable
* if the link object files also DLLIMPORT the same variable.
* For this reason, libpgport can't reference CurrentMemoryContext
* in the palloc macro calls.
*
* To fix this, we create several functions here that allow us to
* manage memory without doing the inline in libpgport.
*/

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +

In pg_config.h:405 this is defined:

/* Define to 1 if you have the <strings.h> header file. */ #define
HAVE_STRINGS_H 1

However, Visual Studio 2005 does not include this file. For a
workaround I simply added it but that's a bit of hack.

Ah, so even though you are using MSVC, you are using the
config file from MinGW. Easy fix, included in the patch
below, and applied for 8.2.

Um, is WIN32_CLIENT_ONLY really defined when you're building a backend
extension? That seems wrong.

The whole problem seems to be the using of the pg_config.h from mingw
when building with msvc. Why not just use the correct pg_config.h in the
first place? What happens then?

IIRC, there were other problems building extensions with MSVC - such as
struct alignment and stuff. Not sure if that has been taken care of? If
not, there really isn't much point in fixing the headers alone :-)

//Magnus

Magnus Hagander wrote:

In pg_config.h:405 this is defined:

/* Define to 1 if you have the <strings.h> header file. */ #define
HAVE_STRINGS_H 1

However, Visual Studio 2005 does not include this file. For a
workaround I simply added it but that's a bit of hack.

Ah, so even though you are using MSVC, you are using the
config file from MinGW. Easy fix, included in the patch
below, and applied for 8.2.

Um, is WIN32_CLIENT_ONLY really defined when you're building a backend
extension? That seems wrong.

Well, it is defined:

#if defined(_MSC_VER) || defined(__BORLANDC__)
#define WIN32_CLIENT_ONLY
#endif

The whole problem seems to be the using of the pg_config.h from mingw
when building with msvc. Why not just use the correct pg_config.h in the
first place? What happens then?

We can't generate a pg_config.h for WIN32_CLIENT_ONLY because there is
no shell build environment.

IIRC, there were other problems building extensions with MSVC - such as
struct alignment and stuff. Not sure if that has been taken care of? If
not, there really isn't much point in fixing the headers alone :-)

Well, I figure alignment would be based on the CPU, not the compiler,
but I might be wrong.

I added the capability, and why it was added, so if we decide the idea
is a failure, we know what to remove.

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +

Um, is WIN32_CLIENT_ONLY really defined when you're

building a backend

extension? That seems wrong.

Well, it is defined:

#if defined(_MSC_VER) || defined(__BORLANDC__)
#define WIN32_CLIENT_ONLY
#endif

Oops. I obviously forgot all about that :-)

The whole problem seems to be the using of the pg_config.h

from mingw

when building with msvc. Why not just use the correct

pg_config.h in

the first place? What happens then?

We can't generate a pg_config.h for WIN32_CLIENT_ONLY because
there is no shell build environment.

Right. But we have include/pg_config.h.win32 which is supposed to be
used in this case. The win32 native build of the backend will copy that
file into pg_config.h.

The problem here is that the backend is built with mingw but the
extension with msvc. If you'd start from fresh, there would be no
pg_config.h to include. The pg_config.h.win32 file is supposed to be
preset for the MSVC environment (which is the same as borland on win32,
but different from mingw).

IIRC, there were other problems building extensions with

MSVC - such

as struct alignment and stuff. Not sure if that has been taken care
of? If not, there really isn't much point in fixing the

headers alone

:-)

Well, I figure alignment would be based on the CPU, not the
compiler, but I might be wrong.

Um. struct packing? I don't remember exactly what it was, but IIRC it
was something.

I added the capability, and why it was added, so if we decide
the idea is a failure, we know what to remove.

Right. I've never tried using msvc for a server extension, and the
general answer has always been "don't do it". But it may well have
changed now.

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

The problem here is that the backend is built with mingw but the
extension with msvc.

I don't think that it's very reasonable to expect that to work.
The two compilers very likely have different rules for struct
packing, to take just the first gotcha. Maybe they are compatible
enough that it will work, but if it does not we cannot fix it.

regards, tom lane

Tom Lane <tgl@sss.pgh.pa.us> writes:

"Magnus Hagander" <mha@sollentuna.net> writes:

The problem here is that the backend is built with mingw but the
extension with msvc.

I don't think that it's very reasonable to expect that to work.
The two compilers very likely have different rules for struct
packing, to take just the first gotcha. Maybe they are compatible
enough that it will work, but if it does not we cannot fix it.

Well that sort of stuff is supposed to be covered by the ABI. Consider that if
it didn't then you wouldn't be able to use any of the standard libraries
without recompiling them for each compiler since a number of standard library
APIs depend on structs like timeval or stat.

I'm not saying there aren't risks, but in theory it's supposed to work.

--
greg

Greg Stark <gsstark@mit.edu> writes:

Well that sort of stuff is supposed to be covered by the ABI. Consider that if
it didn't then you wouldn't be able to use any of the standard libraries
without recompiling them for each compiler since a number of standard library
APIs depend on structs like timeval or stat.

Maybe. My experience with these sorts of problems is fairly old (hard
lessons from libjpeg development in the mid-90s), but what I learned at
the time is that all the standard Unix library interfaces are carefully
designed to be fairly impervious to struct padding rules (eg, all the
fields are "int", or at least all the same size), and so you can get
apparent interoperability between compilers that don't agree on these
rules. Meanwhile, Microsoft, who never met a feature they didn't like,
invented a foot-gun called #pragma pack, and then had to fix all their
headers to nail down the pack setting. That in turn masks compatibility
issues on that platform, but only if you're willing to #pragma-ify all
of your own headers too.

In short, I don't trust anybody on this issue. I have the scars to
remind me why not.

regards, tom lane

Hi everyone,

Thanks for the feedback. The reason for building extensions with MSVC
on windows is to get access to the Micrsoft debugging tools since as far
as I can see MingW/GDB cannot debug dynamically loaded dlls on the
Windows platform (or at least I haven't succeeded at doing it).

Anyway, with the few minor tweaks mentioned this seems to work fine. I
guess I would be surprised if it didn't since MinGW is commonly used for
both creating and using DLLs on the Windows platform that integrate with
programs and dlls built with VC++. Other examples that work fine are
building extensions for Python and Ruby using MingW, when both runtimes
are built with VC++ (so the opposite case). Of course, this
compatibility might just be limited to the MinGW / VC++ combination, but
that's all you need for this to work.

Charlie

Tom Lane wrote:

Charlie Savage wrote:

Hi everyone,

Thanks for the feedback. The reason for building extensions with MSVC
on windows is to get access to the Micrsoft debugging tools since as far
as I can see MingW/GDB cannot debug dynamically loaded dlls on the
Windows platform (or at least I haven't succeeded at doing it).

Anyway, with the few minor tweaks mentioned this seems to work fine. I
guess I would be surprised if it didn't since MinGW is commonly used for
both creating and using DLLs on the Windows platform that integrate with
programs and dlls built with VC++. Other examples that work fine are
building extensions for Python and Ruby using MingW, when both runtimes
are built with VC++ (so the opposite case). Of course, this
compatibility might just be limited to the MinGW / VC++ combination, but
that's all you need for this to work.

OK, I improved the STRINGS_H macro check and added DLLIMPORT in case
Win32 users don't export everything. Added comments. This is all 8.2
stuff.

--
Bruce Momjian http://candle.pha.pa.us
SRA OSS, Inc. http://www.sraoss.com

+ If your life is a hard drive, Christ can be your backup. +