[PATCH] PostgreSQL fails to build with 32bit MinGW-w64

Hi,

Isn't it better to check the value of macros itsef rather than checking for
system dependent macros that does not directly relate to the issue?
specifically for getaddrinfo.c case I think
#if EAI_NODATA != EAI_NONAME
is a better check than checking for
#if !defined(__MINGW64_VERSION_MAJOR) && !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */

For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before
<winsock2.h>
some google suggests that defining _WINSOCKAPI_ before <windows.h> prevents
inclusion of winsock.h but that does not have relation to inclusion of
<winsock2.h> and if <winsock2.h> is included first, it should be ok.

If this guess is right, perhaps it could be better to remove the three lines.
#if !defined(WIN64) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

On 2011/11/24, at 17:24, Lars Kanis wrote:

Hello, Lars.

You wrote:

LK> Hi PostgreSQL hackers,
LK>
LK> support for Mingw-w64 compiler was added to postgres with commit
LK> 91812df. Unfortunately only the 64 bit output is working right
LK> now. This issue was already highlighted with initial patch in
LK> http://archives.postgresql.org/pgsql-bugs/2011-07/msg00059.php
LK>
LK> Mingw-w64 uses the same header files for 32 and 64 bit compiles.
LK> So the same conditions apply to mingw-w32 bit as for the WIN64
LK> case. In WIN64 "WSAAPI" is defined to nothing, but in 32 bit to
LK> stdcall, so it needs to be used in the accept-parameter check,
LK> too. Maybe you prefer PASCAL instead of WSAAPI in configure.
LK>
LK> I tested successful compilation for the following platforms:
LK> - i686-w64-mingw32 - gcc v4.6.1
LK> - x86_64-w64-mingw32 - gcc v4.6.1
LK> - i586-mingw32msvc - gcc v4.4.4
LK> - x86_64-linux-gnu - gcc v4.6.1
LK>

Can you please provide me with some howto on building PG sources with
mingw-w64?

--
With best wishes,
Pavel mailto:pavel@gf.microolap.com

Hi,

2011-11-24 09:55 keltezéssel, Pavel Golub írta:

Can you please provide me with some howto on building PG sources with
mingw-w64?

Install Fedora 15 or 16, add this repo file into /etc/yum.repos.d :

http://build1.openftd.org/fedora-cross/fedora-cross.repo

Then "yum install mingw*". This will install mingw64 both 32 and 64-bit.
Replaces a lot of official mingw32 packages from Fedora.

Then you can do for 32-bit:

cd postgresql-9.1.1
mingw32-configure
mingw32-make
sudo mingw32-make install

or for 64-bit:

mingw64-configure
mingw64-make
sudo mingw64-make install

Best regards,
Zoltán Böszörményi

--
----------------------------------
Zoltán Böszörményi
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt, Austria
Web: http://www.postgresql-support.de
http://www.postgresql.at/

Can you please provide me with some howto on building PG sources with
mingw-w64?

For 32/64 bit mingw-v4.6.1 on ubuntu 11.10:
apt-get install flex gcc-mingw-w64
./configure --host=i686-w64-mingw32 --build=x86_64-linux --without-zlib && make
and
./configure --host=x86_64-w64-mingw32 --build=x86_64-linux --without-zlib && make

For 32 bit mingw-v4.4.4 on ubuntu 11.10:
apt-get install flex gcc-mingw32
./configure --host=i586-mingw32msvc --build=x86_64-linux --without-zlib && make

Regards,
Lars Kanis

Isn't it better to check the value of macros itsef rather than checking for
system dependent macros that does not directly relate to the issue?
specifically for getaddrinfo.c case I think
#if EAI_NODATA != EAI_NONAME
is a better check than checking for
#if !defined(__MINGW64_VERSION_MAJOR) && !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */

Yes it's better and it works for all described test environments.

For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before
<winsock2.h>
some google suggests that defining _WINSOCKAPI_ before <windows.h> prevents
inclusion of winsock.h but that does not have relation to inclusion of
<winsock2.h> and if <winsock2.h> is included first, it should be ok.

If this guess is right, perhaps it could be better to remove the three lines.
#if !defined(WIN64) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif

I only changed this for consistency. For me, it works without that define in all test
environments, too.

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

At least in mingw32 v4.4.4 there is no crtdefs.h. I couldn't find a proper define that relates directly
to that issue, so attached is a somewhat cumbersome MINGW version check.

--
Regards,
Lars Kanis

On 11/24/2011 04:39 AM, Lars Kanis wrote:

Can you please provide me with some howto on building PG sources with

mingw-w64?

For 32/64 bit mingw-v4.6.1 on ubuntu 11.10:

apt-get install flex gcc-mingw-w64

./configure --host=i686-w64-mingw32 --build=x86_64-linux
--without-zlib && make

and

./configure --host=x86_64-w64-mingw32 --build=x86_64-linux
--without-zlib && make

For 32 bit mingw-v4.4.4 on ubuntu 11.10:

apt-get install flex gcc-mingw32

./configure --host=i586-mingw32msvc --build=x86_64-linux
--without-zlib && make

The only thing I at least am going to be interested in supporting is a
native compiler, not a cross-compiler. It's impossible to automate
cross-compiler testing.

The way to build natively with a mingw-w64 compiler is doumented fairly
simply at
<http://www.postgresql.org/docs/current/static/installation-platform-notes.html#INSTALLATION-NOTES-MINGW&gt;:

To build 64 bit binaries using MinGW, install the 64 bit tool set
from http://mingw-w64.sourceforge.net/, put its bin directory in the
PATH, and run configure with the --host=x86_64-w64-mingw option.

This is exactly how the buildfarm member pitta was set up.

The only issue here is how to add support for using the 32bit mingw-w64
compiler in pretty much the same way.

cheers

andrew

Hi,

The way to build natively with a mingw-w64 compiler is doumented fairly simply at <http://www.postgresql.org/docs/current/static/installation-platform-notes.html#INSTALLATION-NOTES-MINGW&gt;:

To build 64 bit binaries using MinGW, install the 64 bit tool set
from http://mingw-w64.sourceforge.net/, put its bin directory in the
PATH, and run configure with the --host=x86_64-w64-mingw option.

This is exactly how the buildfarm member pitta was set up.

The only issue here is how to add support for using the 32bit mingw-w64 compiler in pretty much the same way.

I hope that configure --host=i686-w64-mingw use 32 bit tool sets and
construct a 32 bit binary. It is mostly the same procedure and should have
sufficient information to distinguish them. Both 64 bit and 32 bit binaries
should be runnable on the same 64 bit system.

Best regards,

On 11/24/2011 06:29 AM, Lars Kanis wrote:

Isn't it better to check the value of macros itsef rather than checking for
system dependent macros that does not directly relate to the issue?
specifically for getaddrinfo.c case I think
#if EAI_NODATA != EAI_NONAME
is a better check than checking for
#if !defined(__MINGW64_VERSION_MAJOR)&& !defined(WIN32_ONLY_COMPILER) /* MSVC/WIN64 duplicate */

Yes it's better and it works for all described test environments.

For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before
<winsock2.h>
some google suggests that defining _WINSOCKAPI_ before<windows.h> prevents
inclusion of winsock.h but that does not have relation to inclusion of
<winsock2.h> and if<winsock2.h> is included first, it should be ok.

If this guess is right, perhaps it could be better to remove the three lines.
#if !defined(WIN64) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif

No, this broke some compilers, IIRC (probably the native mingw compiler,
which is in use by several buildfarm members). Getting this right was
very tricky and time-consuming when I was adding support for the 64 bit
mingw-w64 compiler, and there were a couple of rounds of breakage.

I'm therefore much more inclined to go the way of your earlier patch,
which seems much less risky.

I only changed this for consistency. For me, it works without that define in all test
environments, too.

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

Because it's defined by other than mingw-w64 compilers.

We have a bunch of compilers to support here. There are LOTS of compiler
scenarios on Windows (several versions of MSVC, 32bit and 64bit
mingw-w64, native mingw gcc, and a couple of Cygwin based compilers),
and keeping track of them all and making sure they don't break can be a
pain.

cheers

andrew

Hi,

For the win32.h, I really don't understand why _WINSOCKAPI_ was defined before
<winsock2.h>
some google suggests that defining _WINSOCKAPI_ before<windows.h> prevents
inclusion of winsock.h but that does not have relation to inclusion of
<winsock2.h> and if<winsock2.h> is included first, it should be ok.

If this guess is right, perhaps it could be better to remove the three lines.
#if !defined(WIN64) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif

No, this broke some compilers, IIRC (probably the native mingw compiler, which is in use by several buildfarm members). Getting this right was very tricky and time-consuming when I was adding support for the 64 bit mingw-w64 compiler, and there were a couple of rounds of breakage.

I'm therefore much more inclined to go the way of your earlier patch, which seems much less risky.

I agree that original patch could be less risky.
However, it doesn't match what Microsoft says:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms737629%28v=vs.85%29.aspx

So, I think the standard way is not defining _WINSOCKAPI_, and if any compiler requires that,
which I think unlikely, then it should be defined as a exceptional case.
At least, native mingw GCC 4.5.2 (20110802 catalogue) and
4.6.1 (latest catalogue) does compile happily without the three lines.

I only changed this for consistency. For me, it works without that define in all test
environments, too.

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

Because it's defined by other than mingw-w64 compilers.

I see. That's because mingw (not -w64).
Should it be ok if mingw is ok with that condition?

We have a bunch of compilers to support here. There are LOTS of compiler scenarios on Windows (several versions of MSVC, 32bit and 64bit mingw-w64, native mingw gcc, and a couple of Cygwin based compilers), and keeping track of them all and making sure they don't break can be a pain.

Yes, that really is a pain.

The code block
#if _MSC_VER >= 1400 || defined(WIN64)
#define errcode __msvc_errcode
#include <crtdefs.h>
#undef errcode
#endif

looks as if there is no real need to crtdefs.h but, they wanted to prevent
definition of errcode and therefor put to the first place.

So, I was afraid moving the include downwards might break by
by including a header that internally includes crtdefs.h.
If this is not problematic for MSVC (perhaps you know better on that),
I have no objection in moving the order.

Hi,

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

Because it's defined by other than mingw-w64 compilers.

I see. That's because mingw (not -w64).
Should it be ok if mingw is ok with that condition?

This really breaks mingw gcc 4.6.1 :( it does not have crtdefs.h)
If moving downward do not break MSVC, perhaps its the good way.
Otherwise, we might check for the presence of crtdefs.h with configure?

On 11/27/2011 09:18 AM, NISHIYAMA Tomoaki wrote:

Hi,

+/* __MINGW64_VERSION_MAJOR is related to both 32/64 bit gcc compiles by
+ * mingw-w64, however it gots defined only after
Why not use __MINGW32__, which is defined without including any headers?

Because it's defined by other than mingw-w64 compilers.

I see. That's because mingw (not -w64).
Should it be ok if mingw is ok with that condition?

This really breaks mingw gcc 4.6.1 :( it does not have crtdefs.h)
If moving downward do not break MSVC, perhaps its the good way.
Otherwise, we might check for the presence of crtdefs.h with configure?

I have looked at this a bit. It's fairly ugly, and the only moderately
clean way I see forward is a configure test to check for a mingw-w64
compiler, e.g. by running gcc -E -dM over a file which just includes
stdio.h and checking for the definedness of __MINGW64__VERSION_MAJOR, or
something similar.

cheers

andrew

Hi,

A new patch:
check for the presence of crtdefs.h in configure

-#if _MSC_VER >= 1400 || defined(WIN64)
+#if _MSC_VER >= 1400 || HAVE_CRTDEFS_H
#define errcode __msvc_errcode
#include <crtdefs.h>
#undef errcode

Perhaps there is no guarantee that mingw (not -w64) may not have crtdefs.h in the future versions.
the 3 lines
#define errcode __msvc_errcode
#include <crtdefs.h>
#undef errcode
should be valid as far as crtdefs.h exists and errcode is not defined previously.
Because this is the first system include file, we can be sure errcode is not
defined by this point.
So, I believe its better to just test for the presence of crtdefs.h rather than
try to figure out the name of compiler.

check for fseeko and ftello macro definition before defining to avoid
warning in mingw-w64 4.7.0 20110827

This patch was tested to build successfully on
mingw gcc version 4.6.1
mingw-w64 i686-w64-mingw32 gcc version 4.5.4 20110812
mingw-w64 x86_64-w64-mingw32 gcc version 4.7.0 20110827

On Sat, Dec 3, 2011 at 09:24, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

A new patch:
check for the presence of crtdefs.h in configure

-#if _MSC_VER >= 1400 || defined(WIN64)
+#if _MSC_VER >= 1400 || HAVE_CRTDEFS_H
 #define errcode __msvc_errcode
 #include <crtdefs.h>
 #undef errcode

Have you verified if tihs affects _MSC_VER < 1400? Suddently that
branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's
something we need to worry about.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Hi,

Have you verified if tihs affects _MSC_VER < 1400? Suddently that
branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's
something we need to worry about.

I have no MSVC. In that sense it is not verified in fact, and I hope
those who knows well would kindly comment on it.

However, it appears that pg_config.h is not created through
configure, but just copied from pg_config.h.win32 in those
compilers and thus HAVE_CRTDEFS_H will not be defined.
So, I think this code fragment will not be enabled in
_MSC_VER < 1400

In addition, the code fragment should have no harm as far as they
have crtdefs.h.
(Again, this is I think it should, and not tested with real tools.)

On 2011/12/03, at 23:20, Magnus Hagander wrote:

On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

Have you verified if tihs affects _MSC_VER < 1400? Suddently that
branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's
something we need to worry about.

I have no MSVC. In that sense it is not verified in fact, and I hope
those who knows well would kindly comment on it.

However, it appears that pg_config.h is not created through
configure, but just copied from pg_config.h.win32 in those
compilers and thus HAVE_CRTDEFS_H will not be defined.
So, I think this code fragment will not be enabled in
_MSC_VER < 1400

Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it
likely isn't - I was confusing it with the kind of defines that MSVC
tends to stick in their own headerfiles, and thought that's what you
were testing for.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 12/03/2011 09:59 AM, Magnus Hagander wrote:

On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

Have you verified if tihs affects _MSC_VER< 1400? Suddently that
branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's
something we need to worry about.

I have no MSVC. In that sense it is not verified in fact, and I hope
those who knows well would kindly comment on it.

However, it appears that pg_config.h is not created through
configure, but just copied from pg_config.h.win32 in those
compilers and thus HAVE_CRTDEFS_H will not be defined.
So, I think this code fragment will not be enabled in
_MSC_VER< 1400

Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it
likely isn't - I was confusing it with the kind of defines that MSVC
tends to stick in their own headerfiles, and thought that's what you
were testing for.

Yes, but there's a deal more work to do here. This whole thing is
falling over in my build environment (64 bit Windows 7, MinGW/MSys, the
machine that runs pitta on the buildfarm.)

This is a long way from a done deal.

cheers

andrew

On 12/03/2011 06:12 PM, Andrew Dunstan wrote:

On 12/03/2011 09:59 AM, Magnus Hagander wrote:

On Sat, Dec 3, 2011 at 15:49, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

Have you verified if tihs affects _MSC_VER< 1400? Suddently that
branch would care about HAVE_CRTDEFS_H, and I'm not sure if that's
something we need to worry about.

I have no MSVC. In that sense it is not verified in fact, and I hope
those who knows well would kindly comment on it.

However, it appears that pg_config.h is not created through
configure, but just copied from pg_config.h.win32 in those
compilers and thus HAVE_CRTDEFS_H will not be defined.
So, I think this code fragment will not be enabled in
_MSC_VER< 1400

Hmm, true. Unless HAVE_CRTDEFS_H is defined by the sytem, which it
likely isn't - I was confusing it with the kind of defines that MSVC
tends to stick in their own headerfiles, and thought that's what you
were testing for.

Yes, but there's a deal more work to do here. This whole thing is
falling over in my build environment (64 bit Windows 7, MinGW/MSys,
the machine that runs pitta on the buildfarm.)

This is a long way from a done deal.

In particular, it's a major mess because it does this (or at least the
version I'm using does):

#define stat _stat64

which plays merry hell with pgwin32_safestat(). Working around that
looks very unpleasant indeed.

cheers

andrew

Hi,

On 2011/12/04, at 9:45, Andrew Dunstan wrote:

Yes, but there's a deal more work to do here. This whole thing is falling over in my build environment (64 bit Windows 7, MinGW/MSys, the machine that runs pitta on the buildfarm.)

This is a long way from a done deal.

In particular, it's a major mess because it does this (or at least the version I'm using does):

#define stat _stat64

which plays merry hell with pgwin32_safestat(). Working around that looks very unpleasant indeed.

Thank you for testing and reporting.
Would you mind giving me a bit more information on the environment?
--especially for the MinGW/MSYS versions and any other component that is required.

I tested on a virtual machine running 64 bit Windows 7 SP1,
installing MinGW/MSYS in the clean state and then compile.
The versions coming with pre-packaged repository catalogues of 20110802
(gcc 4.5.2) and latest catalogue (gcc-4.6.1-2) compiles successfully.

On 2011/12/04, at 9:45, Andrew Dunstan wrote:

Hi,

I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32)
gcc version 4.7.0 20111203 (experimental) (GCC)

The code can be compiled with

diff --git a/src/include/port.h b/src/include/port.h
index eceb4bf..78d5c92 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir);
  * Some frontends don't need the size from stat, so if UNSAFE_STAT_OK
  * is defined we don't bother with this.
  */
-#if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK)
+#if defined(WIN32_ONLY_COMPILER) && !defined(UNSAFE_STAT_OK)
 #include <sys/stat.h>
 extern int     pgwin32_safestat(const char *path, struct stat * buf);

but, surely we need to know if it is ok or not
as the comment before says:
* stat() is not guaranteed to set the st_size field on win32, so we
* redefine it to our own implementation that is.

Is there any simple test program that determines if the pgwin32_safestat
is required or the library stat is sufficient?
I presume the stat is a library function and therefore it depends on the
compiler rather than the WIN32 platform as a whole.

On 2011/12/04, at 12:55, NISHIYAMA Tomoaki wrote:

On Sun, Dec 4, 2011 at 09:14, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32)
gcc version 4.7.0 20111203 (experimental) (GCC)

The code can be compiled with

diff --git a/src/include/port.h b/src/include/port.h
index eceb4bf..78d5c92 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir);
 * Some frontends don't need the size from stat, so if UNSAFE_STAT_OK
 * is defined we don't bother with this.
 */
-#if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK)
+#if defined(WIN32_ONLY_COMPILER) && !defined(UNSAFE_STAT_OK)
 #include <sys/stat.h>
 extern int     pgwin32_safestat(const char *path, struct stat * buf);

but, surely we need to know if it is ok or not
as the comment before says:
 * stat() is not guaranteed to set the st_size field on win32, so we
 * redefine it to our own implementation that is.

Is there any simple test program that determines if the pgwin32_safestat
is required or the library stat is sufficient?
I presume the stat is a library function and therefore it depends on the
compiler rather than the WIN32 platform as a whole.

No, stat() is unreliable because it is implemented on top of
FindNextFile(), and *that's* where the actual problem is at. And
that's an OS API function, not a library function. See the discussion
at http://archives.postgresql.org/pgsql-hackers/2008-03/msg01181.php

In theory, if mingw implemented their stat() without using
FindNextFile(), it might work - but I don't see how they'd do it in
that case. And I can't see us going into details to remove such a
simple workaround even if they do - it's better to ensure we work the
same way with different compilers.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 12/04/2011 06:31 AM, Magnus Hagander wrote:

On Sun, Dec 4, 2011 at 09:14, NISHIYAMA Tomoaki
<tomoakin@staff.kanazawa-u.ac.jp> wrote:

Hi,

I found error on #define stat _stat64 occurs on Mingw-w64 (x86_64-w64-mingw32)
gcc version 4.7.0 20111203 (experimental) (GCC)

The code can be compiled with

diff --git a/src/include/port.h b/src/include/port.h
index eceb4bf..78d5c92 100644
--- a/src/include/port.h
+++ b/src/include/port.h
@@ -332,7 +332,7 @@ extern bool rmtree(const char *path, bool rmtopdir);
* Some frontends don't need the size from stat, so if UNSAFE_STAT_OK
* is defined we don't bother with this.
*/
-#if defined(WIN32)&&  !defined(__CYGWIN__)&&  !defined(UNSAFE_STAT_OK)
+#if defined(WIN32_ONLY_COMPILER)&&  !defined(UNSAFE_STAT_OK)
#include<sys/stat.h>
extern int     pgwin32_safestat(const char *path, struct stat * buf);

but, surely we need to know if it is ok or not
as the comment before says:
* stat() is not guaranteed to set the st_size field on win32, so we
* redefine it to our own implementation that is.

Is there any simple test program that determines if the pgwin32_safestat
is required or the library stat is sufficient?
I presume the stat is a library function and therefore it depends on the
compiler rather than the WIN32 platform as a whole.

No, stat() is unreliable because it is implemented on top of
FindNextFile(), and *that's* where the actual problem is at. And
that's an OS API function, not a library function. See the discussion
at http://archives.postgresql.org/pgsql-hackers/2008-03/msg01181.php

In theory, if mingw implemented their stat() without using
FindNextFile(), it might work - but I don't see how they'd do it in
that case. And I can't see us going into details to remove such a
simple workaround even if they do - it's better to ensure we work the
same way with different compilers.

Yeah.

This is only a problem because, with this compiler, configure finds this:

checking for _FILE_OFFSET_BITS value needed for large files... 64
checking size of off_t... 8

whereas on the mingw-w64 compiler pitta is using it finds this:

checking for _FILE_OFFSET_BITS value needed for large files... unknown
checking for _LARGE_FILES value needed for large files... unknown
checking size of off_t... 4

It's the setting of _FILE_OFFSET_BITS that causes the offending macro
definition.

Can we just turn off largefile support for this compiler, or maybe for
all mingw builds, possibly by just disabling the checks in configure.in?
I note it's turned off for MSVC in all flavors apparently.
pgwin32_safestat() isn't safe for large files anyway, so there would be
good grounds for doing so quite apart from this, ISTM. (Of course, we
should work out how to handle large files properly on Windows, but
that's a task for another day.)

(BTW, someone asked me the other day why anyone would want to do 32 bit
builds. One answer is that often the libraries you want to link with are
only available in 32 bit versions.)

cheers

andrew

Hi,

If we are not to use 64 bit file size (and time),
#undef stat may be sufficient. The #undef should be
before the prototype of pgwin32_safestat because the
#define stat _stat64 affect both the function and struct stat.
The #undef stat necessitate #undef fstat as the parameter
struct stat * is changed.

Additional change are for the macro redefinition warnings.
(Suppress warnings, but perhaps not very different)

The patch is tested to compile on
x86_64-w64-mingw32-gcc 4.7.0 20111203 (experimental)
and
gcc version 4.6.1 on MingW/MSYS

--- a/src/include/port.h
+++ b/src/include/port.h
@@ -334,6 +334,12 @@ extern bool rmtree(const char *path, bool rmtopdir);
  */
 #if defined(WIN32) && !defined(__CYGWIN__) && !defined(UNSAFE_STAT_OK)
 #include <sys/stat.h>
+#ifdef stat
+#undef stat
+#endif
+#ifdef fstat
+#undef fstat
+#endif
 extern int	pgwin32_safestat(const char *path, struct stat * buf);

#define stat(a,b) pgwin32_safestat(a,b)

If this is not sufficient, we might need to change all call of stat, lstat, and fstat
to some wrapper functions? : It's theoretically doable, but could be quite difficult
for a huge software.

On 12/05/2011 09:31 AM, NISHIYAMA Tomoaki wrote:

Hi,

If we are not to use 64 bit file size (and time),
#undef stat may be sufficient. The #undef should be
before the prototype of pgwin32_safestat because the
#define stat _stat64 affect both the function and struct stat.
The #undef stat necessitate #undef fstat as the parameter
struct stat * is changed.

I don't think I'm going to do it that way, but leave this with me, I can
take it from here. Right now I'm down to the following interesting
regression failure:

    $ cat regression.diffs
    ***
    C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/expected/float8-exp-three-digits-win32.out   
    Fri Nov 25 14:24:49 2011
    ---
    C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/results/float8.out   
    Mon Dec  5 18:17:36 2011
    ***************
    *** 382,388 ****
          SET f1 = FLOAT8_TBL.f1 * '-1'
          WHERE FLOAT8_TBL.f1 > '0.0';
       SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f;
    ! ERROR:  value out of range: overflow
       SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f;
       ERROR:  value out of range: overflow
       SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5;
    --- 382,396 ----
          SET f1 = FLOAT8_TBL.f1 * '-1'
          WHERE FLOAT8_TBL.f1 > '0.0';
       SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f;
    !  bad |     ?column?
    ! -----+------------------
    !      |                0
    !      |      -3.484e+201
    !      |     -1.0043e+203
    !      |        -Infinity
    !      | -1.2345678901234
    ! (5 rows)
    !
       SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f;
       ERROR:  value out of range: overflow
       SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5;

======================================================================

cheers

andrew

On 12/05/2011 06:27 PM, Andrew Dunstan wrote:

$ cat regression.diffs
***

C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/expected/float8-exp-three-digits-win32.out
Fri Nov 25 14:24:49 2011
---

C:/MinGW/msys/1.0/home/pgrunner/bf/root32/HEAD/pgsql/src/test/regress/results/float8.out   
Mon Dec  5 18:17:36 2011
***************
*** 382,388 ****
SET f1 = FLOAT8_TBL.f1 * '-1'
WHERE FLOAT8_TBL.f1 > '0.0';
SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f;
! ERROR:  value out of range: overflow
SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f;
ERROR:  value out of range: overflow
SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5;
--- 382,396 ----
SET f1 = FLOAT8_TBL.f1 * '-1'
WHERE FLOAT8_TBL.f1 > '0.0';
SELECT '' AS bad, f.f1 * '1e200' from FLOAT8_TBL f;
!  bad |     ?column?
! -----+------------------
!      |                0
!      |      -3.484e+201
!      |     -1.0043e+203
!      |        -Infinity
!      | -1.2345678901234
! (5 rows)
!
SELECT '' AS bad, f.f1 ^ '1e200' from FLOAT8_TBL f;
ERROR:  value out of range: overflow
SELECT 0 ^ 0 + 0 ^ 1 + 0 ^ 0.0 + 0 ^ 0.5;

======================================================================

This is apparently an optimization bug in the compiler. If I turn
optimization off (CFLAGS=-O0) it goes away. Ick.

So at the moment I'm a bit blocked. I can't really file a bug because
the compiler can't currently be used to build postgres, I don't have
time to construct a self-contained test case, and I don't want to commit
changes to enable the compiler until the issue is solved.

FYI I've been testing with the attached patch. We'll need to construct a
configure test for HAVE_CRTDEFS_H.

cheers

andrew

Hi,

FYI I've been testing with the attached patch.
We'll need to construct a configure test for HAVE_CRTDEFS_H.

Isn't it enough to add the name in configure.in and run autoconf to
update configure and autoheaders to update pg_config.h.in?
The check of win32 before large file perhaps should also go to configure.in,
otherwise they would be wiped with next autoconf.

The patch recreated with removing the #undef
but adding the conditional to skip AC_SYS_LARGEFILE in configure.in and
update configure by autoconf.

On Thu, Dec 8, 2011 at 12:46 PM, Andrew Dunstan <andrew@dunslane.net> wrote:

This is apparently an optimization bug in the compiler. If I turn
optimization off (CFLAGS=-O0) it goes away. Ick.

So at the moment I'm a bit blocked. I can't really file a bug because the
compiler can't currently be used to build postgres, I don't have time to
construct a self-contained test case, and I don't want to commit changes to
enable the compiler until the issue is solved.

If we're talking about adding support for a previously-unsupported
configuration, it seems to me that it would be fine to commit a patch
that made everything work, but for the compiler bug. We could refrain
from stating that we officially support that configuration until the
compiler bug is fixed, or even document the existence of the bug. We
can't be responsible for other people's broken code, but I don't
necessarily see that as a reason not to commit a prerequisite patch.
Otherwise, as you say, there's a chicken-and-egg problem, and who does
that help?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On 12/09/2011 03:11 PM, Robert Haas wrote:

On Thu, Dec 8, 2011 at 12:46 PM, Andrew Dunstan<andrew@dunslane.net> wrote:

This is apparently an optimization bug in the compiler. If I turn
optimization off (CFLAGS=-O0) it goes away. Ick.

So at the moment I'm a bit blocked. I can't really file a bug because the
compiler can't currently be used to build postgres, I don't have time to
construct a self-contained test case, and I don't want to commit changes to
enable the compiler until the issue is solved.

If we're talking about adding support for a previously-unsupported
configuration, it seems to me that it would be fine to commit a patch
that made everything work, but for the compiler bug. We could refrain
from stating that we officially support that configuration until the
compiler bug is fixed, or even document the existence of the bug. We
can't be responsible for other people's broken code, but I don't
necessarily see that as a reason not to commit a prerequisite patch.
Otherwise, as you say, there's a chicken-and-egg problem, and who does
that help?

Yeah, fair enough. I'll work on that.

cheers

andrew

On -10/01/37 20:59, Andrew Dunstan wrote:

This is apparently an optimization bug in the compiler. If I turn
optimization off (CFLAGS=-O0) it goes away. Ick.

So at the moment I'm a bit blocked. I can't really file a bug because
the
compiler can't currently be used to build postgres, I don't have time to
construct a self-contained test case, and I don't want to commit
changes to
enable the compiler until the issue is solved.

If we're talking about adding support for a previously-unsupported
configuration, it seems to me that it would be fine to commit a patch
that made everything work, but for the compiler bug. We could refrain
from stating that we officially support that configuration until the
compiler bug is fixed, or even document the existence of the bug. We
can't be responsible for other people's broken code, but I don't
necessarily see that as a reason not to commit a prerequisite patch.
Otherwise, as you say, there's a chicken-and-egg problem, and who does
that help?

Yeah, fair enough. I'll work on that.

Definitely do this (and then file a bug report with the project). I've
worked with both Kai and NightStrike from the MingW-W64 project to fix
previous bugs, and as long as they can build the offending source
themselves then they are very helpful and quick to respond.

ATB,

Mark.

--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063

Sirius Labs: http://www.siriusit.co.uk/labs

Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan:

Yeah, fair enough. I'll work on that.

Many thanks for reviewing, tweaking and commiting the patch!
One thing I wonder about, is this snippet. Is the define really needed now?

* The Mingw64 headers choke if this is already defined - they
* define it themselves.
*/
#if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif
#include <winsock2.h>

Kind regards,
Lars Kanis

On 12/12/2011 06:43 AM, Mark Cave-Ayland wrote:

configuration, it seems to me that it would be fine to commit a patch
that made everything work, but for the compiler bug. We could refrain
from stating that we officially support that configuration until the
compiler bug is fixed, or even document the existence of the bug. We
can't be responsible for other people's broken code, but I don't
necessarily see that as a reason not to commit a prerequisite patch.
Otherwise, as you say, there's a chicken-and-egg problem, and who does
that help?

Yeah, fair enough. I'll work on that.

If we're talking about adding support for a previously-unsupported

Definitely do this (and then file a bug report with the project). I've
worked with both Kai and NightStrike from the MingW-W64 project to fix
previous bugs, and as long as they can build the offending source
themselves then they are very helpful and quick to respond.

Done and done (see
<https://sourceforge.net/tracker/?func=detail&amp;aid=3458244&amp;group_id=202880&amp;atid=983354&gt;).

cheers

andrew

On 12/12/2011 09:54 AM, Lars Kanis wrote:

Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan:

Yeah, fair enough. I'll work on that.

Many thanks for reviewing, tweaking and commiting the patch!
One thing I wonder about, is this snippet. Is the define really needed now?

* The Mingw64 headers choke if this is already defined - they
* define it themselves.
*/
#if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif
#include<winsock2.h>

As previously discussed, unless you can prove it's not needed I don't
want to remove it, on the ""if it ain't broke don't fix it" principle. I
believe it is needed for at least some older compilers (specifically
some of those used by buildfarm animals narwhal, frogmouth, mastodon,
hamerkop and currawong), and it doesn't appear to be hurting anything.
As you can see above it's been disabled for all Mingw-w64 compilers.

If it's really bugging people we can try disabling it and see if any of
those break, but honestly we have far uglier things that we carry for
legacy reasons :-)

cheers

andrew

Am Montag, 12. Dezember 2011, 10:19:46 schrieb Andrew Dunstan:

On 12/12/2011 09:54 AM, Lars Kanis wrote:

Am Freitag, 9. Dezember 2011, 15:31:17 schrieb Andrew Dunstan:

Yeah, fair enough. I'll work on that.

Many thanks for reviewing, tweaking and commiting the patch!
One thing I wonder about, is this snippet. Is the define really needed now?

* The Mingw64 headers choke if this is already defined - they
* define it themselves.
*/
#if !defined(__MINGW64_VERSION_MAJOR) || defined(WIN32_ONLY_COMPILER)
#define _WINSOCKAPI_
#endif
#include<winsock2.h>

As previously discussed, unless you can prove it's not needed I don't
want to remove it, on the ""if it ain't broke don't fix it" principle. I
believe it is needed for at least some older compilers (specifically
some of those used by buildfarm animals narwhal, frogmouth, mastodon,
hamerkop and currawong), and it doesn't appear to be hurting anything.
As you can see above it's been disabled for all Mingw-w64 compilers.

Ok. Thanks for clarification.

Kind regards,
Lars

On 12/14/2011 04:43 AM, Mark Cave-Ayland wrote:

On 12/12/11 15:00, Andrew Dunstan wrote:

Yeah, fair enough. I'll work on that.

If we're talking about adding support for a previously-unsupported

Definitely do this (and then file a bug report with the project). I've
worked with both Kai and NightStrike from the MingW-W64 project to fix
previous bugs, and as long as they can build the offending source
themselves then they are very helpful and quick to respond.

Done and done (see
<https://sourceforge.net/tracker/?func=detail&amp;aid=3458244&amp;group_id=202880&amp;atid=983354&gt;).

Hi Andrew,

Did you see Kai's update on the ticket? If this is the case, I know
that we have seen similar bugs with PostGIS and the work-around has
been to add -ffloat-store to the compiler flags for the affected files
if that helps?

Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the
error goes away.

So, would we want to use that just for this file, or for the whole build?

FTR, the comment in the bug reads:

AFAICS from report, the issue happens with value 1e200 (as invalid
range).
The issue I might see here (as it doesn't occure with x64 version, which
uses sse instructions instead of x87) is that x87 registers
internally have
higher precision then 32-bit. So failures in range occure on conversion
from FPU register down to memory store. For x64 SSE this is
different, as
here math operations are really just done in specified precission.
Have you checked, if you get different result by using on 32-bit
explicit
SSE instructions?

As things seems to work at -O0, but not at -On (with n > 0), it is
pretty
unlikely that runtime-functions itself are causing this issue. So
therefore my guess goes here for internal/external precision of used
FPU.

cheers

andrew

On 14/12/11 13:59, Andrew Dunstan wrote:

Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the
error goes away.

So, would we want to use that just for this file, or for the whole build?

Well the latest documentation for gcc gives 2 options: -ffloat-store and
-fexcess-precision=<style> which are documented at
http://gcc.gnu.org/onlinedocs/gcc-4.6.2/gcc/Optimize-Options.html#Optimize-Options.

For PostGIS we only applied the flags for specific files, because of
severe performance warnings in older versions of the gcc documentation
such as this: http://www.delorie.com/gnu/docs/gcc/gcc_10.html. I have no
idea whether these warnings still hold true or not for more modern
compiler versions.

ISTM that the best solution would be to determine whether or not
-fexcess-precision=standard does the right thing (and also determine the
performance hit) or take a look at the "excess precision" notes in the
older documentation to see if parts of the code can be rearranged to
eliminate the effect.

ATB,

Mark.

--
Mark Cave-Ayland - Senior Technical Architect
PostgreSQL - PostGIS
Sirius Corporation plc - control through freedom
http://www.siriusit.co.uk
t: +44 870 608 0063

Sirius Labs: http://www.siriusit.co.uk/labs

Andrew Dunstan <andrew@dunslane.net> writes:

Done and done (see
<https://sourceforge.net/tracker/?func=detail&amp;aid=3458244&amp;group_id=202880&amp;atid=983354&gt;).

Did you see Kai's update on the ticket? If this is the case, I know
that we have seen similar bugs with PostGIS and the work-around has
been to add -ffloat-store to the compiler flags for the affected files
if that helps?

Hmm. Yeah, if I remove -O0 and instead set CFLAGS to -ffloat-store the
error goes away.

Hmm, we have been bit by that recently elsewhere:
http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=ff68b256a533b398e3420750f34d161aeee4e099

I suspect what you are looking at is not at all mingw-specific but will
soon start showing up on other x86 platforms. I see from the bug report
that that's gcc 4.7.0, which hasn't made it into most distros yet but
surely will soon.

So, would we want to use that just for this file, or for the whole build?

-ffloat-store is a brute force solution, I think, and would affect old
versions of gcc that don't exhibit any problems. I would suggest
altering configure to see whether the compiler recognizes
-fexcess-precision=standard and adding that to CFLAGS if so.

regards, tom lane

On Wed, Dec 14, 2011 at 11:14 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

-ffloat-store is a brute force solution, I think, and would affect old
versions of gcc that don't exhibit any problems.  I would suggest
altering configure to see whether the compiler recognizes
-fexcess-precision=standard and adding that to CFLAGS if so.

Would it be better to change either the code or the test case to be
less sensitive to this issue?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:

On Wed, Dec 14, 2011 at 11:14 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

-ffloat-store is a brute force solution, I think, and would affect old
versions of gcc that don't exhibit any problems. I would suggest
altering configure to see whether the compiler recognizes
-fexcess-precision=standard and adding that to CFLAGS if so.

Would it be better to change either the code or the test case to be
less sensitive to this issue?

AFAICS it's really impractical to do that. The code Andrew is having
problems with is essentially

double a,b,c;
...
a = b * c;
if (isinf(a)) throw error;

and the problem is that the multiplication result overflows in double
precision, but not in the wider-than-double register precision.
Therefore, if a is in a register and the isinf() primitive inspects the
register, it will return false, even though when the value gets stored
to memory it will become an infinity.

I don't see anything we can do to the code that avoids this issue.
You might think that explicitly casting b * c to double would help,
but our experiments in connection with the planner Assert case proved
it didn't. The only other thing we could possibly do is move the
multiplication into a separate subroutine, but what's to stop the
compiler from inlining that and generating the same code anyway?

Basically, what's going on here is that the gcc boys have decided
that speed trumps both sanity and conformance to the letter of the C
standard, unless you turn on compiler switches that say "please act
sane". So we'd better do that, unless you'd like to be dealing with
this type of issue for the rest of the project's lifespan. It's much
the same type of problem as with -fno-strict-aliasing, except that
someday we might consider biting the bullet and dealing with that piece
of insanity-in-the-name-of-speed. Floating-point performance is not
interesting enough for Postgres' purposes that I can imagine that we'd
ever want to deal with this kind of gotcha to improve FP speed.

regards, tom lane

On Wed, Dec 14, 2011 at 12:36 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

AFAICS it's really impractical to do that.  The code Andrew is having
problems with is essentially

       double a,b,c;
       ...
       a = b * c;
       if (isinf(a)) throw error;

and the problem is that the multiplication result overflows in double
precision, but not in the wider-than-double register precision.
Therefore, if a is in a register and the isinf() primitive inspects the
register, it will return false, even though when the value gets stored
to memory it will become an infinity.

Uh, wow. That really is pretty insane. How is anyone supposed to
write sensible code around that non-API?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Robert Haas <robertmhaas@gmail.com> writes:

Uh, wow. That really is pretty insane. How is anyone supposed to
write sensible code around that non-API?

Usability seems to be very low on the gcc project's list of goals
these days.

Personally I think this sort of thing might be fine if it were triggered
by -ffast-math or something like that. But as a default behavior it's
entirely ridiculous.

regards, tom lane

On 12/14/2011 11:14 AM, Tom Lane wrote:

-ffloat-store is a brute force solution, I think, and would affect old
versions of gcc that don't exhibit any problems. I would suggest
altering configure to see whether the compiler recognizes
-fexcess-precision=standard and adding that to CFLAGS if so.

OK, this and the associated configure change seems to do the trick:

    diff --git a/configure.in b/configure.in
    index 9cf084d..b29bb61 100644
    --- a/configure.in
    +++ b/configure.in
    @@ -437,6 +437,8 @@ if test "$GCC" = yes -a "$ICC" = no; then
        PGAC_PROG_CC_CFLAGS_OPT([-fno-strict-aliasing])
        # Disable optimizations that assume no overflow; needed for gcc 4.3+
        PGAC_PROG_CC_CFLAGS_OPT([-fwrapv])
    +  # Disable FP optimizations that cause isinf errors on gcc 4.5+
    +  PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard])
      elif test "$ICC" = yes; then
        # Intel's compiler has a bug/misoptimization in checking for
        # division by NAN (NaN == 0), -mp1 fixes it, so add it to the
    CFLAGS.

I guess we should backpatch it?

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

+  # Disable FP optimizations that cause isinf errors on gcc 4.5+
+  PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard])

Looks sane to me, except "isinf errors" is an awfully narrow reading of
the problem. Maybe just say "assorted errors"? Also, do we know that
gcc 4.5 poses the issue? I'm only aware of reports for 4.6 and 4.7.

I guess we should backpatch it?

+1. Back branches will see these same problems as soon as anybody
tries to compile them with latest-n-greatest gcc.

regards, tom lane

On 12/14/2011 03:09 PM, Tom Lane wrote:

Andrew Dunstan<andrew@dunslane.net> writes:

+  # Disable FP optimizations that cause isinf errors on gcc 4.5+
+  PGAC_PROG_CC_CFLAGS_OPT([-fexcess-precision=standard])

Looks sane to me, except "isinf errors" is an awfully narrow reading of
the problem. Maybe just say "assorted errors"? Also, do we know that
gcc 4.5 poses the issue? I'm only aware of reports for 4.6 and 4.7.

It looked to me like this switch landed in gcc 4.5 because they were
getting problems like this. See
<http://gcc.gnu.org/ml/gcc-patches/2008-11/msg00105.html&gt;

I guess we should backpatch it?

+1. Back branches will see these same problems as soon as anybody
tries to compile them with latest-n-greatest gcc.

Yeah. Will do.

cheers

andrew