libpq sockets on win32

I found that sending more than 16k of data in a SQL statement caused a
problem. I was using PyGreSQL but narrowed the error down to
"\postgresql-7.1.2\src\interfaces\libpq\fe-misc.c" pqReadData() and
pqFlush(). After some poking around, it looked like recv was setting
errno to ENOENT for some reason. I figured out why today. Win32
sockets don't set errno at all. ENOENT was just left in errno from
some earlier call.

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I can't read that web site under Netscape.

I haven't done much C coding in a few years and don't want to break
other code by blindly doing:

#define errno WSAGetLastError

If I could read it, I think I could fix it. Please send it in some
readable format.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I can't read that web site under Netscape.

If I could read it, I think I could fix it. Please send it in some
readable format.

I chopped the content out and stuck it into a basic HTML file and
attached it.

Thanks for taking a look,
Jeff

Can you try this patch and let me know if it helps? It is a different
approach. This was the only place I saw errno checked for a
non-predefined value.

One other solution may be to use the #define only in the libpq C files
that need it. What really stinks is that the errno define is only
useful for socket errno settings.

I do see a use in fe-connect.c:

#ifndef WIN32
if (errno == EINPROGRESS || errno == 0)
#else
if (WSAGetLastError() == WSAEINPROGRESS)
#endif

I hate to litter this through the whole source. I wonder if we have to
bracket the errno checkes with #define/#undef. Can you try that with
the fix described on the web page. The above would convert to:

#ifdef WIN32
#define errno WSAGetLastError
#endif
if (errno == EINPROGRESS || errno == 0)
#ifdef WIN32
#undef errno
#endif

Maybe make these into their own macros somehow.

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I can't read that web site under Netscape.

If I could read it, I think I could fix it. Please send it in some
readable format.

I chopped the content out and stuck it into a basic HTML file and
attached it.

Thanks for taking a look,
Jeff

[ Attachment, skipping... ]

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Jeff Johnson writes:

After some poking around, it looked like recv was setting errno to
ENOENT for some reason. I figured out why today. Win32 sockets don't
set errno at all. ENOENT was just left in errno from some earlier
call.

There are already a few (wrong) attempts to cover this situation in
fe-connect.c, but it looks like a bunch of #ifdef's are needed.

I haven't done much C coding in a few years and don't want to break
other code by blindly doing:

#define errno WSAGetLastError

Probably rather

#ifndef WIN32
normal code;
#else
windows code;
#endif

--
Peter Eisentraut peter_e@gmx.net http://funkturm.homeip.net/~peter

Bruce Momjian <pgman@candle.pha.pa.us> writes:

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I can't read that web site under Netscape.

Comes up fine for me (maybe you need to turn off Javascript?)

However, the advice on it seems to be written by someone who does not
know the difference between a variable and a function :-(. Read with
a large grain of salt.

We already have a couple of #ifdef'd usages of WSAGetLastError() in
libpq, but it's quite unclear which other references to errno might
need to be changed. A blanket replacement of errno would certainly
break things.

regards, tom lane

Bruce Momjian <pgman@candle.pha.pa.us> writes:

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I can't read that web site under Netscape.

Comes up fine for me (maybe you need to turn off Javascript?)

Thanks, that fixed it.

However, the advice on it seems to be written by someone who does not
know the difference between a variable and a function :-(. Read with
a large grain of salt.

We already have a couple of #ifdef'd usages of WSAGetLastError() in
libpq, but it's quite unclear which other references to errno might
need to be changed. A blanket replacement of errno would certainly
break things.

That's why I recommended brackets of #define/#undef around tests of
socket errno's.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

Bruce Momjian wrote:

Can you try this patch and let me know if it helps? It is a

different

approach. This was the only place I saw errno checked for a
non-predefined value.

Setting errno = 0 doesn't help, the error handling code is entered
when recv returns -1, then even if errno == 0, it'll bomb out.

I hate to litter this through the whole source. I wonder if
we have to
bracket the errno checkes with #define/#undef. Can you try that

with

the fix described on the web page. The above would convert to:

#ifdef WIN32
#define errno WSAGetLastError
#endif
if (errno == EINPROGRESS || errno == 0)
#ifdef WIN32
#undef errno
#endif

Maybe make these into their own macros somehow.

Even when I was a C programmer I never did much more than simple
defines with the pre-compiler so I'll leave this to those that know
how :)

As Tom Lane points out in another post, the "define errno
WSAGetLastError" seems to confuse a variable with a function. I was
surprised that such a thing could work. I'm happy to hear that it
doesn't.

What about something like this:

#ifdef WIN32
#define s_errno WSAGetLastError()
#else
#define s_errno errno
#endif

/* for socket functions, check s_errno */
if (s_errno == EINPROGRESS || s_errno == 0)
...

/* for non-socket functions, check errno as usual */
if (errno == ENOENT)
...

Then replace only errno with s_errno when it is used with socket code.
I'm not sure if strerror would work with all the errors returned by
WSAGetLastError(). The Win32 SDK says to use FormatMessage(a ton of
stuff here).

Regards,
Jeff

As Tom Lane points out in another post, the "define errno
WSAGetLastError" seems to confuse a variable with a function. I was
surprised that such a thing could work. I'm happy to hear that it
doesn't.

What about something like this:

#ifdef WIN32
#define s_errno WSAGetLastError()
#else
#define s_errno errno
#endif

/* for socket functions, check s_errno */
if (s_errno == EINPROGRESS || s_errno == 0)
...

/* for non-socket functions, check errno as usual */
if (errno == ENOENT)
...

I have done exactly that. I assume fcntl(), ioctl(), select() use errno
even if used on a socket, while getsockopt(), setsockopt(), socket(),
connect(), getsockname(), send(), recv() use WSAGetLastError. Is this
list correct?

The patch is attached. Please let me know so I can finalize it and
apply it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

I have done exactly that. I assume fcntl(), ioctl(),
select() use errno
even if used on a socket, while getsockopt(), setsockopt(),

socket(),

connect(), getsockname(), send(), recv() use WSAGetLastError. Is

this

list correct?

I don't know enough about such things.

The patch is attached. Please let me know so I can finalize it and
apply it.

Couple of changes required to compile on Win32:

Change:
#define sockerrno WSAGetLastError
To:
#define sockerrno WSAGetLastError()

This has to go back into fe-connect.c, EINPROGRESS isn't defined in
Win32 for some reason..
#ifndef WIN32
if (errno == EINPROGRESS || errno == 0)
#else
if (WSAGetLastError() == WSAEINPROGRESS)
#endif

I tested it out but I'm getting this error when sending a large SQL
statement (>16k).

File "WebKit\Application.py", line 340, in dispatchRequest
self.handleGoodURL(transaction)
File "WebKit\Application.py", line 457, in handleGoodURL
self.respond(transaction)
File "WebKit\Application.py", line 569, in respond
transaction.respond()
File "H:\webware\Webware\WebKit\Transaction.py", line 93, in respond
self._servlet.respond(self)
File "H:\webware\Webware\WebKit\HTTPServlet.py", line 38, in respond
method(self, trans)
File "H:\webware\Webware\WebKit\Page.py", line 34, in respondToGet
self._respond(transaction)
File "H:\webware\Webware\WebKit\Page.py", line 67, in _respond
self.writeHTML()
File "H:\webware\Webware\WebKit\Page.py", line 129, in writeHTML
self.writeBody()
File "H:\webware\Webware\WebKit\Page.py", line 186, in writeBody
self.writeBodyParts()
File "/EA\www\SitePage.py", line 305, in writeBodyParts
self.writeContent()
File "\EA\www\MyTest.py", line 9, in writeContent
self.transferRecords()
File "\EA\www\MyTest.py", line 45, in transferRecords
cu.execute(sql)
File "c:\python20\pgdb.py", line 189, in execute
self.executemany(operation, (params,))
File "c:\python20\pgdb.py", line 204, in executemany
rows = self.__source.execute(sql)
ValueError: pqReadData() -- read() failed: errno=10035
Unknown error

I have done exactly that. I assume fcntl(), ioctl(),
select() use errno
even if used on a socket, while getsockopt(), setsockopt(),

socket(),

connect(), getsockname(), send(), recv() use WSAGetLastError. Is

this

list correct?

I don't know enough about such things.

The web page wasn't clear about that.

The patch is attached. Please let me know so I can finalize it and
apply it.

Couple of changes required to compile on Win32:

Change:
#define sockerrno WSAGetLastError
To:
#define sockerrno WSAGetLastError()

OK.

This has to go back into fe-connect.c, EINPROGRESS isn't defined in
Win32 for some reason..
#ifndef WIN32
if (errno == EINPROGRESS || errno == 0)
#else
if (WSAGetLastError() == WSAEINPROGRESS)
#endif

OK. Not sure why it wasn't defined, but, oh well.

I tested it out but I'm getting this error when sending a large SQL
statement (>16k).
cu.execute(sql)
File "c:\python20\pgdb.py", line 189, in execute
self.executemany(operation, (params,))
File "c:\python20\pgdb.py", line 204, in executemany
rows = self.__source.execute(sql)
ValueError: pqReadData() -- read() failed: errno=10035
Unknown error

That errno is very high, 10035. If I take a look at my sys/errno.h on
BSD/OS, I see:

#define EAGAIN 35 /* Resource temporarily unavailable */

so my guess is that WSAGetLastError() returns errno plus 10,000. What
value does your include file set for EAGAIN, 10035 or 35. If it is 35,
the following patch may work. It has all your suggested fixes.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

I found this in WinSock.h ... it might shed some light on the problem.

#define WSABASEERR 10000
[snip]

/*
* Windows Sockets definitions of regular Berkeley error constants
*/
#define WSAEWOULDBLOCK (WSABASEERR+35)
#define WSAEINPROGRESS (WSABASEERR+36)

-----Original Message-----
From: Bruce Momjian [mailto:pgman@candle.pha.pa.us]
Sent: Monday, June 04, 2001 4:02 PM
To: jeff@jeffjohnson.net
Cc: pgsql-interfaces@postgresql.org; pgsql-docs@postgresql.org
Subject: Re: [INTERFACES] libpq sockets on win32

I have done exactly that. I assume fcntl(), ioctl(),
select() use errno
even if used on a socket, while getsockopt(), setsockopt(),

socket(),

connect(), getsockname(), send(), recv() use WSAGetLastError.

Is

this

list correct?

I don't know enough about such things.

The web page wasn't clear about that.

The patch is attached. Please let me know so I can

finalize it and

apply it.

Couple of changes required to compile on Win32:

Change:
#define sockerrno WSAGetLastError
To:
#define sockerrno WSAGetLastError()

OK.

This has to go back into fe-connect.c, EINPROGRESS isn't defined

in

Win32 for some reason..
#ifndef WIN32
if (errno == EINPROGRESS || errno == 0)
#else
if (WSAGetLastError() == WSAEINPROGRESS)
#endif

OK. Not sure why it wasn't defined, but, oh well.

I tested it out but I'm getting this error when sending a large

SQL

statement (>16k).
cu.execute(sql)
File "c:\python20\pgdb.py", line 189, in execute
self.executemany(operation, (params,))
File "c:\python20\pgdb.py", line 204, in executemany
rows = self.__source.execute(sql)
ValueError: pqReadData() -- read() failed: errno=10035
Unknown error

That errno is very high, 10035. If I take a look at my sys/errno.h

on

BSD/OS, I see:

#define EAGAIN 35 /* Resource temporarily
unavailable */

so my guess is that WSAGetLastError() returns errno plus 10,000.

What

I found this in WinSock.h ... it might shed some light on the problem.

#define WSABASEERR 10000
[snip]

/*
* Windows Sockets definitions of regular Berkeley error constants
*/
#define WSAEWOULDBLOCK (WSABASEERR+35)
#define WSAEINPROGRESS (WSABASEERR+36)

I modified the patch to use WSABASEERR rather than 10000.

Do your includes define EAGAIN and stuff like that? Whare are those
values? If they are the same as WSAEAGAIN, then my patch should work.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

I have added this thread to TODO.detail/libpq.

I found that sending more than 16k of data in a SQL statement caused a
problem. I was using PyGreSQL but narrowed the error down to
"\postgresql-7.1.2\src\interfaces\libpq\fe-misc.c" pqReadData() and
pqFlush(). After some poking around, it looked like recv was setting
errno to ENOENT for some reason. I figured out why today. Win32
sockets don't set errno at all. ENOENT was just left in errno from
some earlier call.

This article describes the problem and work around.
http://msdn.microsoft.com/library/psdk/winsock/ovrvw3_26ia.htm

I haven't done much C coding in a few years and don't want to break
other code by blindly doing:

#define errno WSAGetLastError

Does anyone that knows this code want to take a crack at it?

Thanks,
Jeff

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026