Another crack at doing a Win32 build under MINGW

(guess) try configuring without readline.

cheers

andrew

Dann Corbit wrote:

U�ytkownik Andrew Dunstan napisa�:

(guess) try configuring without readline.

Or compile readline with your Mingw version, or just replace readline.a
with readline.dll in your lib directory.
All works with readline fine.

Best regards
Rony

I am able to build now, and perform initdb. However, I cannot run the
postmaster. I don't know how far along the port is. What is the
current state of the port to Win32?

dcorbit@DANNFAST /usr/local/pgsql/bin
$ postmaster -D u:/pgdata
LOG: select() failed in postmaster: No such file or directory

dcorbit@DANNFAST /usr/local/pgsql/bin
$ FATAL: could not attach to proper memory at fixed address:
shmget(key=5432001, addr=00E10000) failed: No such file or directory

Dann Corbit wrote:

I am able to build now, and perform initdb. However, I cannot run the
postmaster. I don't know how far along the port is. What is the
current state of the port to Win32?

dcorbit@DANNFAST /usr/local/pgsql/bin
$ postmaster -D u:/pgdata
LOG: select() failed in postmaster: No such file or directory

dcorbit@DANNFAST /usr/local/pgsql/bin
$ FATAL: could not attach to proper memory at fixed address:
shmget(key=5432001, addr=00E10000) failed: No such file or directory

[ email moved to win32 list.]

They have only a few regression tests failing, so we are very far along.

The message you are seeing looks like code that assumes that a child can
map to the same shared memory address as the postmaster. We haven't
seen that fail for anyone before, but it is an assumption we weren't
sure about. Of course this is all a guess.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

I've seen both these messages after each other when -i is not
specified. Been meaning to adress the issue of it not failing
gracefully without -i on win32.

Anyway. It seems the postmaster goes down while a child process is
still going up (stats collector, I guess) or something along that
line. This way the child can't attach to shared memory, and

there you

go.

If you add PID information to the log, you will notice that the
messages are from two different processes.

Is there a case for forcing -i and ignoring the GUC setting
on Windows? Since we can't do Unix domain sockets there it
would seem to make sense.

Yeah, that could be done. I was more into doing a generic fix that would
fail gracefully in any case when the server is not listening on anything
(no Unix, no TCPIP) and error out then.

Are there any other platforms which don't have unix sockets? If not,
then that thought is not valid, and we shuold just force it on win32. If
not, how do they handle starting of the postmaster without -i today? And
do we want the same behaviour there?

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

//Magnus

Magnus Hagander said:

I've seen both these messages after each other when -i is not
specified. Been meaning to adress the issue of it not failing
gracefully without -i on win32.

Anyway. It seems the postmaster goes down while a child process is
still going up (stats collector, I guess) or something along that
line. This way the child can't attach to shared memory, and

there you

go.

If you add PID information to the log, you will notice that the
messages are from two different processes.

Is there a case for forcing -i and ignoring the GUC setting
on Windows? Since we can't do Unix domain sockets there it
would seem to make sense.

Yeah, that could be done. I was more into doing a generic fix that
would fail gracefully in any case when the server is not listening on
anything (no Unix, no TCPIP) and error out then.

Are there any other platforms which don't have unix sockets? If not,
then that thought is not valid, and we shuold just force it on win32.
If not, how do they handle starting of the postmaster without -i today?
And do we want the same behaviour there?

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

Hmm. That also raises the question of what we should do if virtual_host is
set.

[thinks some more ...]

cheers

andrew

I wrote:

Magnus Hagander said:

Is there a case for forcing -i and ignoring the GUC setting
on Windows? Since we can't do Unix domain sockets there it
would seem to make sense.

Yeah, that could be done. I was more into doing a generic fix that
would fail gracefully in any case when the server is not listening on
anything (no Unix, no TCPIP) and error out then.

Are there any other platforms which don't have unix sockets? If not,
then that thought is not valid, and we shuold just force it on win32.
If not, how do they handle starting of the postmaster without -i today?
And do we want the same behaviour there?

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

Hmm. That also raises the question of what we should do if virtual_host is
set.

[thinks some more ...]

.....

How does this sound?

. if -i/tcpip_socket is not set, then bind to localhost
. if -i/tcpip_socket is set, and virtual_host is not set, behave as now
(i.e. bind to all addresses)
. if -i/tcpip_socket is set, and virtual_host is set, bind to all but
immediately close connections where the local address is not either
localhost or the virtual_host.

That seems to me to get as close as reasonably possible to the Unix
behaviour. I don't think that always allowing localhost connections on
Windows is a big security risk.

Also, what is the default connection mode of psql? It should probably be
equivalent to "-h localhost", shouldn't it?

I haven't thought through what might be the IP4/IP6 implications.

cheers

andrew

"Magnus Hagander" <mha@sollentuna.net> writes:

Are there any other platforms which don't have unix sockets?

There are a couple, although I think those ports may be moribund
(if anyone still cares about BeOS, they haven't mentioned it lately).

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

That seems like a reasonable compromise.

regards, tom lane

Tom Lane wrote:

"Magnus Hagander" <mha@sollentuna.net> writes:

Are there any other platforms which don't have unix sockets?

There are a couple, although I think those ports may be moribund
(if anyone still cares about BeOS, they haven't mentioned it lately).

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

That seems like a reasonable compromise.

Agreed, but how do we do that. pg_hba.conf already only listens on
127.0.0.1. Do we ignore non-local IPs in that file until they use -i?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

Tom Lane wrote:

"Magnus Hagander" <mha@sollentuna.net> writes:

Are there any other platforms which don't have unix sockets?

There are a couple, although I think those ports may be moribund
(if anyone still cares about BeOS, they haven't mentioned it lately).

Perhaps we should force it to open a tcp socket on 127.0.0.1 only? That
way we don't suddenly open up to external connections without the user
asking for it.

That seems like a reasonable compromise.

Agreed, but how do we do that. pg_hba.conf already only listens on
127.0.0.1. Do we ignore non-local IPs in that file until they use -i?

Those are remote addresses, not local addresses. pg_hba.conf doesn't say
anything at all about the listening address. The errors that have been
reported would have passed the default pg_hba.conf filters - the problem
as I understand it was that there was no listener on the localhost
interface.

It is true, though,. that you can't (or shouldn't) be able to connect to
localhost except from localhost.

cheers

andrew

Andrew Dunstan wrote:

How does this sound?

. if -i/tcpip_socket is not set, then bind to localhost
. if -i/tcpip_socket is set, and virtual_host is not set, behave as now
(i.e. bind to all addresses)
. if -i/tcpip_socket is set, and virtual_host is set, bind to all but
immediately close connections where the local address is not either
localhost or the virtual_host.

That seems to me to get as close as reasonably possible to the Unix
behaviour. I don't think that always allowing localhost connections on
Windows is a big security risk.

Also, what is the default connection mode of psql? It should probably be
equivalent to "-h localhost", shouldn't it?

Now that is something I had not thought of. Seems we can assume a Win32
psql can never use unix domain sockets, so defaulting that to localhost
is a good solution too.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Andrew Dunstan wrote:

That seems to me to get as close as reasonably possible to the Unix
behaviour. I don't think that always allowing localhost connections on
Windows is a big security risk.

Is it a big security risk anywhere? Perhaps there is a case to be made
that on all platforms, "-i" should enable or disable only nonlocal
connections. Without -i we'd only allow binding to loopback ports
(either IP4 or IP6).

Aside from keeping the Windows and Unix behaviors similar, this would be
of some positive benefit for people who use TCP-only clients. They'd
not have to remember to set -i anymore, unless they want remote access.

In response to Andrew's table, here's what I'm visualizing:

* No -i: bind only to loopback addresses (both IP4 and IP6 if available).
* With -i, but not virtual_host: bind to all available addresses.
* With -i and virtual_host: bind to specified address(es) only.

(Note this is orthogonal to pg_hba.conf checks; we are talking about
what socket addresses the postmaster listens on.)

I don't have a strong feeling about the case of virtual_host without -i.
The above says to ignore virtual_host, but maybe we should instead
ignore the lack of -i and do what virtual_host says.

regards, tom lane

Tom Lane wrote:

Perhaps there is a case to be made
that on all platforms, "-i" should enable or disable only nonlocal
connections. Without -i we'd only allow binding to loopback ports
(either IP4 or IP6).

Aside from keeping the Windows and Unix behaviors similar, this would be
of some positive benefit for people who use TCP-only clients. They'd
not have to remember to set -i anymore, unless they want remote access.

I've been caught by this more than once, because I use local JDBC
clients, so now the first thing I do after initdb is to set tcpip_socket
to true.

In response to Andrew's table, here's what I'm visualizing:

* No -i: bind only to loopback addresses (both IP4 and IP6 if available).

OK.

* With -i, but not virtual_host: bind to all available addresses.

OK.

* With -i and virtual_host: bind to specified address(es) only.

Can you bind one socket to more than one address? My understanding is
that it's all (IN_ADDR_ANY or in6_addr_any) or one. In that case with
this proposal we'd have to force all the communications through that
interface on Windows. Or would we use multiple sockets (in which case
there is probably a good case for allowing multiple addresses in
virtual_host)?

... (haven't toured this part of the code before) ....

I see what looks like an array of listen sockets, so multiple sockets
seems the way to go.

I don't have a strong feeling about the case of virtual_host without -i.
The above says to ignore virtual_host, but maybe we should instead
ignore the lack of -i and do what virtual_host says.

I have no strong feelings either.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

* With -i and virtual_host: bind to specified address(es) only.

Can you bind one socket to more than one address?

Nope. That's why we have provisions to listen on more than one socket.
I think that was implemented for virtual_host, but it'd be necessary
anyway for dual IPv4/IPv6 support.

regards, tom lane

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

* With -i and virtual_host: bind to specified address(es) only.

Can you bind one socket to more than one address?

Nope. That's why we have provisions to listen on more than one socket.
I think that was implemented for virtual_host, but it'd be necessary
anyway for dual IPv4/IPv6 support.

I should have read the code instead of the docs, which only talk about
one address under virtual_host, not a space separated list of them:

virtual_host (string)

Specifies the host name or IP address on which the server is to
listen for connections from client applications. The default is to
listen on all configured addresses (including localhost).

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

I should have read the code instead of the docs, which only talk about
one address under virtual_host, not a space separated list of them:

That's pretty bogus. I've committed improved wording.

regards, tom lane

[removing to hackers as it is of general interest]

Tom Lane wrote:

Andrew Dunstan wrote:

That seems to me to get as close as reasonably possible to the Unix
behaviour. I don't think that always allowing localhost connections on
Windows is a big security risk.

Is it a big security risk anywhere? Perhaps there is a case to be made
that on all platforms, "-i" should enable or disable only nonlocal
connections. Without -i we'd only allow binding to loopback ports
(either IP4 or IP6).

Aside from keeping the Windows and Unix behaviors similar, this would be
of some positive benefit for people who use TCP-only clients. They'd
not have to remember to set -i anymore, unless they want remote access.

In response to Andrew's table, here's what I'm visualizing:

* No -i: bind only to loopback addresses (both IP4 and IP6 if available).
* With -i, but not virtual_host: bind to all available addresses.
* With -i and virtual_host: bind to specified address(es) only.

(Note this is orthogonal to pg_hba.conf checks; we are talking about
what socket addresses the postmaster listens on.)

I don't have a strong feeling about the case of virtual_host without -i.
The above says to ignore virtual_host, but maybe we should instead
ignore the lack of -i and do what virtual_host says.

This slipped off my radar. I have just spent a little while thinking
about it. How about this: we replace tcpip_socket and virtual_host with
a new var called listen_addresses, which can have values of "local",
"all", or a list of addresses? The default would be "local" and -i would
correspond to "all".

Yes, I know it's not backwards compatible, but we just went through that
argument with log_line_prefix ;-)

Actually, if we wanted to go the whole hog with virtual hosting we'd
allow per-address port specification, like apache does, but maybe that's
something to be left for another day ;-)

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

This slipped off my radar. I have just spent a little while thinking
about it. How about this: we replace tcpip_socket and virtual_host with
a new var called listen_addresses, which can have values of "local",
"all", or a list of addresses? The default would be "local" and -i would
correspond to "all".

No objection here. You could also allow "none" (or maybe that falls out
of the list case by writing an empty list), with the understanding that
"none" is a useless setting on Windows or any other platform that
doesn't support Unix sockets.

Yes, I know it's not backwards compatible, but we just went through that
argument with log_line_prefix ;-)

I think it's the same argument: neither of these variables are likely to
be touched by application code, only by config file entries; so we need
not feel compelled to provide backwards-compatibility options.

Actually, if we wanted to go the whole hog with virtual hosting we'd
allow per-address port specification, like apache does, but maybe that's
something to be left for another day ;-)

Yeah, I don't think that's worth the trouble. It would open up a bunch
of definitional issues (like which port number do we use as the seed for
the shared memory key), without actually buying much useful functionality.

regards, tom lane

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

This slipped off my radar. I have just spent a little while thinking
about it. How about this: we replace tcpip_socket and virtual_host with
a new var called listen_addresses, which can have values of "local",
"all", or a list of addresses? The default would be "local" and -i would
correspond to "all".

No objection here. You could also allow "none" (or maybe that falls out
of the list case by writing an empty list), with the understanding that
"none" is a useless setting on Windows or any other platform that
doesn't support Unix sockets.

I had thought about "none". I have no strong feelings either way,
althought you are right that it would be redundant.

On platforms without Unix domain sockets I think we should immediately
error exit without a valid tcp listening address.

I will try to get this done in the next few weeks - I think it's
critical for sensible Windows use.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

How about this: we replace tcpip_socket and virtual_host with
a new var called listen_addresses, which can have values of "local",
"all", or a list of addresses? The default would be "local" and -i would
correspond to "all".

No objection here. You could also allow "none" (or maybe that falls out
of the list case by writing an empty list), with the understanding that
"none" is a useless setting on Windows or any other platform that
doesn't support Unix sockets.

I had thought about "none". I have no strong feelings either way,
althought you are right that it would be redundant.

Also, "local" would be better spelled "localhost", which would make it
not a special case but just the same as writing a list containing just
that one name. That leaves only "all" as a special case. Perhaps you
could write "all" as "*", and thereby not have any special keywords
needed in the variable: it's either "*" or a list of addresses or names.
The advantage of this is you need not worry about collisions of DNS
names with keywords.

regards, tom lane

Bruce Momjian wrote:

Andrew Dunstan wrote:

Also, what is the default connection mode of psql? It should probably be
equivalent to "-h localhost", shouldn't it?

Now that is something I had not thought of. Seems we can assume a Win32
psql can never use unix domain sockets, so defaulting that to localhost
is a good solution too.

The trivial patch below does this (I think). I still don't have an
available Windows box for testing, so someone who does please verify.

thanks

andrew

Index: src/bin/psql/startup.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/psql/startup.c,v
retrieving revision 1.85
diff -c -r1.85 startup.c
*** src/bin/psql/startup.c 19 Feb 2004 19:40:09 -0000 1.85
--- src/bin/psql/startup.c 15 Mar 2004 10:37:50 -0000
***************
*** 153,158 ****
--- 153,162 ----
pset.getPassword = false;
#endif

+ #ifndef HAVE_UNIX_SOCKETS
+ options.host = "localhost";
+ #endif
+
parse_psql_options(argc, argv, &options);

if (!pset.popt.topt.fieldSep)

Andrew, Tom:

This will be a really nice feature for those of us with PG servers that
participate in VPNs. Currently I'm blocking certain interfaces using
pg_hba.conf but would prefer a "listen" address instead.

Of course, the drawback to this is that confused DBAs will have their
pg_hba.conf conflict with their postgresql.conf, and cut off all access to
the DB. But I don't know how we can protect against that.

Might I suggest that this default to "127.0.0.1" in postgresql.conf.sample?
This is a reasonably safe default, and would allow us to use the same default
for Windows as for other OSes. It would also eliminate about 15% of the
questions I get on a weekly basis from PHP users. ("uncomment the line
tcpip_sockets ...").

If I had time, I would also love to see setting the password for the postgres
user become part of the initdb script. However, I can see that this wouldn't
work with packages.

--
Josh Berkus
Aglio Database Solutions
San Francisco

Josh Berkus wrote:

Andrew, Tom:

This will be a really nice feature for those of us with PG servers that
participate in VPNs. Currently I'm blocking certain interfaces using
pg_hba.conf but would prefer a "listen" address instead.

Of course, the drawback to this is that confused DBAs will have their
pg_hba.conf conflict with their postgresql.conf, and cut off all access to
the DB. But I don't know how we can protect against that.

Might I suggest that this default to "127.0.0.1" in postgresql.conf.sample?
This is a reasonably safe default, and would allow us to use the same default
for Windows as for other OSes. It would also eliminate about 15% of the
questions I get on a weekly basis from PHP users. ("uncomment the line
tcpip_sockets ...").

If I had time, I would also love to see setting the password for the postgres
user become part of the initdb script. However, I can see that this wouldn't
work with packages.

Why couldn't we do something where we ask for a password only if stdin
is from a terminal?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Josh Berkus <josh@agliodbs.com> writes:

Might I suggest that this default to "127.0.0.1" in postgresql.conf.sample?

No, the default should be "localhost". Your thinking is too
IPv4-centric.

regards, tom lane

Josh Berkus wrote:

Andrew, Tom:

This will be a really nice feature for those of us with PG servers that
participate in VPNs. Currently I'm blocking certain interfaces using
pg_hba.conf but would prefer a "listen" address instead.

You can configure listening addresses now using the virtual_host GUC
setting. The documentation was execrable, but Tom has fixed that.

Of course, the drawback to this is that confused DBAs will have their
pg_hba.conf conflict with their postgresql.conf, and cut off all access to
the DB. But I don't know how we can protect against that.

That surely can't be more than are seen now on IRC who can't contact
their DBs because they forgot to turn on tcpip. Plus this does not
intefere at all with Unix sockets, so they should still be able to use
the local psql (except on Windows, where you have to use tcpip sockets).
I am betting that 95%+ of users will either use the default (no remote
connections) or "*" (bind to all interfaces).

Might I suggest that this default to "127.0.0.1" in postgresql.conf.sample?
This is a reasonably safe default, and would allow us to use the same default
for Windows as for other OSes. It would also eliminate about 15% of the
questions I get on a weekly basis from PHP users. ("uncomment the line
tcpip_sockets ...").

The intention is to make "localhost" the default. That should translate
to 127.0.0.1 and ::1 (if they have ipv6 on). Of course, if they have a
broken resolver things might get sticky, but that is true now anyway.

If I had time, I would also love to see setting the password for the postgres
user become part of the initdb script. However, I can see that this wouldn't
work with packages.

Orthogonal problem.

cheers

andrew

Tom,

No, the default should be "localhost". Your thinking is too
IPv4-centric.

Good point. My clients are all years away from implementing Ipv6, so I tend
to forget about it.

--
-Josh Berkus
Aglio Database Solutions
San Francisco

I wrote:

Josh Berkus wrote:

If I had time, I would also love to see setting the password for the
postgres user become part of the initdb script. However, I can see
that this wouldn't work with packages.

Orthogonal problem.

BTW, initdb is no longer a script - some idiot rewrote it in C :-)

And it does have a switch for setting the superuser password:

-W
--pwprompt

Makes initdb prompt for a password to give the database superuser.
If you don't plan on using password authentication, this is not
important. Otherwise you won't be able to use password
authentication until you have a password set up.

cheers

andrew

Idiot is such a strong word, don't you think?

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org
[mailto:pgsql-hackers-owner@postgresql.org] On Behalf Of Andrew Dunstan
Sent: Tuesday, March 16, 2004 7:26 AM
To: Postgresql Hackers
Subject: Re: [HACKERS] listening addresses

I wrote:

Josh Berkus wrote:

If I had time, I would also love to see setting the password for the
postgres user become part of the initdb script. However, I can see
that this wouldn't work with packages.

Orthogonal problem.

BTW, initdb is no longer a script - some idiot rewrote it in C :-)

And it does have a switch for setting the superuser password:

-W
--pwprompt

Makes initdb prompt for a password to give the database superuser.
If you don't plan on using password authentication, this is not
important. Otherwise you won't be able to use password
authentication until you have a password set up.

cheers

andrew

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if
your
joining column's datatypes do not match

Andrew Dunstan <andrew@dunslane.net> writes:

The intention is to make "localhost" the default. That should translate
to 127.0.0.1 and ::1 (if they have ipv6 on). Of course, if they have a
broken resolver things might get sticky, but that is true now anyway.

Just to be clear: right now, if "localhost" doesn't resolve then the
stats collector will not start, but this doesn't prevent use of the
database. It might be a good idea to ensure that the same is still true
after making this change; that is, invalid entries in listen_addresses
should only provoke warnings and not refusal to start (assuming that
we are able to find at least one valid socket to listen to, of course).
I believe that right now, any bad entry in virtual_host causes the
postmaster to error out. That's defensible from one point of view but
on balance I think it's overly paranoid. Any other opinions out there?

regards, tom lane

John Hansen wrote:

Idiot is such a strong word, don't you think?

you should hear what my enemies call me ....

cheers

andrew

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

The intention is to make "localhost" the default. That should translate
to 127.0.0.1 and ::1 (if they have ipv6 on). Of course, if they have a
broken resolver things might get sticky, but that is true now anyway.

Just to be clear: right now, if "localhost" doesn't resolve then the
stats collector will not start, but this doesn't prevent use of the
database. It might be a good idea to ensure that the same is still true
after making this change; that is, invalid entries in listen_addresses
should only provoke warnings and not refusal to start (assuming that
we are able to find at least one valid socket to listen to, of course).
I believe that right now, any bad entry in virtual_host causes the
postmaster to error out. That's defensible from one point of view but
on balance I think it's overly paranoid. Any other opinions out there?

Makes sense - we are not giving anything away but *not* listening to
something.

I did wonder if we should treate "localhost" as a bit special and not
rely on the resolver for it.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

I did wonder if we should treate "localhost" as a bit special and not
rely on the resolver for it.

I don't think so; we went in the other direction in 7.4 for pgstats.
(It used to try to bind to "127.0.0.1" and now tries "localhost".)
So far I've not seen any evidence that makes me think that was a bad
choice.

regards, tom lane

Tom Lane wrote:

Josh Berkus <josh@agliodbs.com> writes:

Might I suggest that this default to "127.0.0.1" in postgresql.conf.sample?

No, the default should be "localhost". Your thinking is too
IPv4-centric.

FYI, once we default to listening on localhost, we need to warn folks
who are using socket permission to control access that they have to turn
off localhost. That needs to be mentioned in the release notes, and in
the SGML docs that talk about socket permissions.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

FYI, once we default to listening on localhost, we need to warn folks
who are using socket permission to control access that they have to turn
off localhost. That needs to be mentioned in the release notes, and in
the SGML docs that talk about socket permissions.

True. I don't see this as a big problem, since use of socket
permissions requires some nondefault settings in postgresql.conf
anyway (IIRC). Adjusting listen_addresses too shouldn't be a big
deal.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

FYI, once we default to listening on localhost, we need to warn folks
who are using socket permission to control access that they have to turn
off localhost. That needs to be mentioned in the release notes, and in
the SGML docs that talk about socket permissions.

True. I don't see this as a big problem, since use of socket
permissions requires some nondefault settings in postgresql.conf
anyway (IIRC). Adjusting listen_addresses too shouldn't be a big
deal.

No, it isn't a problem. I just wanted to mention is so we remember to
address it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

I did wonder if we should treate "localhost" as a bit special and not
rely on the resolver for it.

I don't think so; we went in the other direction in 7.4 for pgstats.
(It used to try to bind to "127.0.0.1" and now tries "localhost".)
So far I've not seen any evidence that makes me think that was a bad
choice.

A small problem with it was reported to me a couple of days ago - user
had firewalled off all IP6 traffic. The stats collector happily bound
and connected to the socket, but all the packets fell in the bit bucket.
They found it quite hard to diagnose the problem.

Possible solutions that occurred to me:
. an initial "hello"-"yes i'm here" exchange to validate the address
. a configurable stats collector address
. fix the firewall ("Doctor, it hurts when I move like this." - "So,
stop moving like that.")

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

A small problem with it was reported to me a couple of days ago - user
had firewalled off all IP6 traffic. The stats collector happily bound
and connected to the socket, but all the packets fell in the bit bucket.
They found it quite hard to diagnose the problem.

Possible solutions that occurred to me:
. an initial "hello"-"yes i'm here" exchange to validate the address

That one seems reasonable to me. Seems like it would take just a few
more lines of code in the loop that tries to find a working socket to
check that we can send a byte and receive it. You'd have to be careful
not to block if the socket is indeed not working ... also, is it safe to
assume that a byte sent with send() is *immediately* ready to recv()?

regards, tom lane

Tom Lane wrote:

also, is it safe to
assume that a byte sent with send() is *immediately* ready to recv()?

If not presumably you could either sleep for a very small interval
before the recv or select on the socket for a very small interval. Half
a second should be ample, I would think.

This doesn't strike me as a high priority item, but possibly worth
putting on the TODO list? Or I could just include it when I get around
to the rest of the listening address stuff we discussed earlier.

cheers

andrew

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

A small problem with it was reported to me a couple of days ago - user
had firewalled off all IP6 traffic. The stats collector happily bound
and connected to the socket, but all the packets fell in the bit bucket.
They found it quite hard to diagnose the problem.

Possible solutions that occurred to me:
. an initial "hello"-"yes i'm here" exchange to validate the address

That one seems reasonable to me. Seems like it would take just a few
more lines of code in the loop that tries to find a working socket to
check that we can send a byte and receive it. You'd have to be careful
not to block if the socket is indeed not working ... also, is it safe to
assume that a byte sent with send() is *immediately* ready to recv()?

This patch attempts to implement the idea, with safety in case the
packet is not immediately available.

comments welcome

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

This patch attempts to implement the idea, with safety in case the
packet is not immediately available.

Seems like you ought to be testing for failure returns from send() and
recv(). Also, what of EINTR from select()?

regards, tom lane

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

This patch attempts to implement the idea, with safety in case the
packet is not immediately available.

Seems like you ought to be testing for failure returns from send() and
recv().

Good point. will do.

Also, what of EINTR from select()?

It will fail. Not sure what else it should do - I'm open to suggestions.
Retry?

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

Tom Lane wrote:

Also, what of EINTR from select()?

It will fail. Not sure what else it should do - I'm open to suggestions.
Retry?

Yes. AFAIR, all our other select calls will just loop indefinitely on
EINTR.

regards, tom lane

Andrew Dunstan wrote:

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

A small problem with it was reported to me a couple of days ago -
user had firewalled off all IP6 traffic. The stats collector happily
bound and connected to the socket, but all the packets fell in the
bit bucket. They found it quite hard to diagnose the problem.

Possible solutions that occurred to me:
. an initial "hello"-"yes i'm here" exchange to validate the address

That one seems reasonable to me. Seems like it would take just a few
more lines of code in the loop that tries to find a working socket to
check that we can send a byte and receive it. You'd have to be careful
not to block if the socket is indeed not working ... also, is it safe to
assume that a byte sent with send() is *immediately* ready to recv()?

Revised patch attached. I think this is about as much trouble as this
problem is worth ;-)

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

A small problem with it was reported to me a couple of days ago -
user had firewalled off all IP6 traffic. The stats collector happily
bound and connected to the socket, but all the packets fell in the
bit bucket. They found it quite hard to diagnose the problem.

Revised patch attached. I think this is about as much trouble as this
problem is worth ;-)

I thought the messages were a bit sloppy, which made the patch much less
useful than it should be: we are testing for a very specific failure
mode and we can give a very specific message. Patch as-applied is
attached.

I don't have any real convenient way to set up a situation where this
failure can actually occur. Anyone want to verify that the patch
acts as intended?

regards, tom lane

*** src/backend/postmaster/pgstat.c.orig	Mon Mar 15 15:01:57 2004
--- src/backend/postmaster/pgstat.c	Mon Mar 22 18:55:29 2004
***************
*** 191,196 ****
--- 191,202 ----
  			   *addr,
  				hints;
  	int			ret;
+ 	fd_set      rset;
+ 	struct timeval tv;
+ 	char        test_byte;
+ 	int         sel_res;
+ 
+ #define TESTBYTEVAL ((char) 199)

  	/*
  	 * Force start of collector daemon if something to collect
***************
*** 303,308 ****
--- 309,393 ----
  			ereport(LOG,
  					(errcode_for_socket_access(),
  					 errmsg("could not connect socket for statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 
+ 		/*
+ 		 * Try to send and receive a one-byte test message on the socket.
+ 		 * This is to catch situations where the socket can be created but
+ 		 * will not actually pass data (for instance, because kernel packet
+ 		 * filtering rules prevent it).
+ 		 */
+ 		test_byte = TESTBYTEVAL;
+ 		if (send(pgStatSock, &test_byte, 1, 0) != 1)
+ 		{
+ 			ereport(LOG,
+ 					(errcode_for_socket_access(),
+ 					 errmsg("could not send test message on socket for statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 
+ 		/*
+ 		 * There could possibly be a little delay before the message can be
+ 		 * received.  We arbitrarily allow up to half a second before deciding
+ 		 * it's broken.
+ 		 */
+ 		for (;;)				/* need a loop to handle EINTR */
+ 		{
+ 			FD_ZERO(&rset);
+ 			FD_SET(pgStatSock, &rset);
+ 			tv.tv_sec = 0;
+ 			tv.tv_usec = 500000;
+ 			sel_res = select(pgStatSock+1, &rset, NULL, NULL, &tv);
+ 			if (sel_res >= 0 || errno != EINTR)
+ 				break;
+ 		}
+ 		if (sel_res < 0)
+ 		{
+ 			ereport(LOG,
+ 					(errcode_for_socket_access(),
+ 					 errmsg("select() failed in statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 		if (sel_res == 0 || !FD_ISSET(pgStatSock, &rset))
+ 		{
+ 			/*
+ 			 * This is the case we actually think is likely, so take pains to
+ 			 * give a specific message for it.
+ 			 *
+ 			 * errno will not be set meaningfully here, so don't use it.
+ 			 */
+ 			ereport(LOG,
+ 					(ERRCODE_CONNECTION_FAILURE,
+ 					 errmsg("test message did not get through on socket for statistics collector")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 
+ 		test_byte++;			/* just make sure variable is changed */
+ 
+ 		if (recv(pgStatSock, &test_byte, 1, 0) != 1)
+ 		{
+ 			ereport(LOG,
+ 					(errcode_for_socket_access(),
+ 					 errmsg("could not receive test message on socket for statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 
+ 		if (test_byte != TESTBYTEVAL) /* strictly paranoia ... */
+ 		{
+ 			ereport(LOG,
+ 					(ERRCODE_INTERNAL_ERROR,
+ 					 errmsg("incorrect test message transmission on socket for statistics collector")));
  			closesocket(pgStatSock);
  			pgStatSock = -1;
  			continue;

Patch applied by Tom. Thanks.

---------------------------------------------------------------------------

Andrew Dunstan wrote:

Andrew Dunstan wrote:

Tom Lane wrote:

Andrew Dunstan <andrew@dunslane.net> writes:

A small problem with it was reported to me a couple of days ago -
user had firewalled off all IP6 traffic. The stats collector happily
bound and connected to the socket, but all the packets fell in the
bit bucket. They found it quite hard to diagnose the problem.

Possible solutions that occurred to me:
. an initial "hello"-"yes i'm here" exchange to validate the address

That one seems reasonable to me. Seems like it would take just a few
more lines of code in the loop that tries to find a working socket to
check that we can send a byte and receive it. You'd have to be careful
not to block if the socket is indeed not working ... also, is it safe to
assume that a byte sent with send() is *immediately* ready to recv()?

Revised patch attached. I think this is about as much trouble as this
problem is worth ;-)

cheers

andrew

Index: src/backend/postmaster/pgstat.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/backend/postmaster/pgstat.c,v
retrieving revision 1.61
diff -c -r1.61 pgstat.c
*** src/backend/postmaster/pgstat.c	15 Mar 2004 16:21:37 -0000	1.61
--- src/backend/postmaster/pgstat.c	22 Mar 2004 12:47:54 -0000
***************
*** 191,196 ****
--- 191,200 ----
*addr,
hints;
int			ret;
+ 	fd_set      rset;
+ 	struct timeval tv;
+ 	char        test_byte;
+ 	int         sel_res;

/*
* Force start of collector daemon if something to collect
***************
*** 307,312 ****
--- 311,365 ----
pgStatSock = -1;
continue;
}
+ 
+ #define TESTBYTEVAL 99
+ 
+ 		/* try to send and receive a test byte on the socket */
+ 
+ 		test_byte = TESTBYTEVAL;
+ 		if (send(pgStatSock,&test_byte,1,0) != 1)
+ 		{
+ 			ereport(LOG,
+ 					(errcode_for_socket_access(),
+ 					 errmsg("test byte send failure socket for statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 		test_byte++;
+ 		for(;;)
+ 		{
+ 			FD_ZERO(&rset);
+ 			FD_SET(pgStatSock, &rset);
+ 			tv.tv_sec = 0;
+ 			tv.tv_usec = 500000;
+ 			sel_res = select(pgStatSock+1,&rset,NULL,NULL,&tv);
+ 			if (sel_res != -1 || errno != EINTR)
+ 				break;
+ 		}
+ 		if( sel_res > 0 && FD_ISSET(pgStatSock,&rset) 
+ 			&& recv(pgStatSock,&test_byte,1,0) == 1)
+ 		{
+ 			if(test_byte != TESTBYTEVAL)
+ 			{
+ 				ereport(LOG,
+ 						(errcode_for_socket_access(),
+ 						 errmsg("incorrect test value on send/receive on socket for statistics collector")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 			}
+ 		}
+ 		else
+ 		{
+ 			ereport(LOG,
+ 					(errcode_for_socket_access(),
+ 					 errmsg("could receive test byte on socket for statistics collector: %m")));
+ 			closesocket(pgStatSock);
+ 			pgStatSock = -1;
+ 			continue;
+ 		}
+ 		

/* If we get here, we have a working socket */
break;

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Patch applied, with comment added. Thanks.

---------------------------------------------------------------------------

Andrew Dunstan wrote:

Bruce Momjian wrote:

Andrew Dunstan wrote:

Also, what is the default connection mode of psql? It should probably be
equivalent to "-h localhost", shouldn't it?

Now that is something I had not thought of. Seems we can assume a Win32
psql can never use unix domain sockets, so defaulting that to localhost
is a good solution too.

The trivial patch below does this (I think). I still don't have an
available Windows box for testing, so someone who does please verify.

thanks

andrew

Index: src/bin/psql/startup.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/psql/startup.c,v
retrieving revision 1.85
diff -c -r1.85 startup.c
*** src/bin/psql/startup.c 19 Feb 2004 19:40:09 -0000 1.85
--- src/bin/psql/startup.c 15 Mar 2004 10:37:50 -0000
***************
*** 153,158 ****
--- 153,162 ----
pset.getPassword = false;
#endif

+ #ifndef HAVE_UNIX_SOCKETS
+ options.host = "localhost";
+ #endif
+
parse_psql_options(argc, argv, &options);

if (!pset.popt.topt.fieldSep)

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Should we have used 127.0.0.1 rather than 'localhost' here?

---------------------------------------------------------------------------

Andrew Dunstan wrote:

Bruce Momjian wrote:

Andrew Dunstan wrote:

Also, what is the default connection mode of psql? It should probably be
equivalent to "-h localhost", shouldn't it?

Now that is something I had not thought of. Seems we can assume a Win32
psql can never use unix domain sockets, so defaulting that to localhost
is a good solution too.

The trivial patch below does this (I think). I still don't have an
available Windows box for testing, so someone who does please verify.

thanks

andrew

Index: src/bin/psql/startup.c
===================================================================
RCS file: /projects/cvsroot/pgsql-server/src/bin/psql/startup.c,v
retrieving revision 1.85
diff -c -r1.85 startup.c
*** src/bin/psql/startup.c 19 Feb 2004 19:40:09 -0000 1.85
--- src/bin/psql/startup.c 15 Mar 2004 10:37:50 -0000
***************
*** 153,158 ****
--- 153,162 ----
pset.getPassword = false;
#endif

+ #ifndef HAVE_UNIX_SOCKETS
+ options.host = "localhost";
+ #endif
+
parse_psql_options(argc, argv, &options);

if (!pset.popt.topt.fieldSep)

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Should we have used 127.0.0.1 rather than 'localhost' here?

No. Think IPv6.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Should we have used 127.0.0.1 rather than 'localhost' here?

No. Think IPv6.

OK, just asking in case.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073