Win32 & NLS

Magnus Hagander wrote:

Which appears to suggest that we should change the locale using
putenv() etc, and not using setlocale() at all... Because setlocale()
does not support LC_MESSAGES, probably.

This cannot possibly work. putenv() doesn't change any locale. The
environment variables only serve as a default when you call setlocale()
without an actual locale value. But to enable any locale, you always
need to call setlocale() somehow, somewhere.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Which appears to suggest that we should change the locale using
putenv() etc, and not using setlocale() at all... Because setlocale()
does not support LC_MESSAGES, probably.

This cannot possibly work. putenv() doesn't change any locale. The
environment variables only serve as a default when you call
setlocale()
without an actual locale value. But to enable any locale, you always
need to call setlocale() somehow, somewhere.

That was my first thought, too. But it *does* appear to work.

With this patch, I can change lc_messages from
environment-before-startup, from postgresql.conf or using "set
lc_messages='foo'".
The only main thing I see as being wrong is that it cannot check if the
specified messages were incorrect. If I literally set it to "foo", I get
english without getting any error/warning.

Clearly they don't just serve as defaults on win32. That's the only
conclusion I can draw from the fact that this works, and from the
questions in the FAQ referenced.

See also for example
http://lists.gnu.org/archive/html/bug-gnu-utils/2004-02/msg00091.html

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

However, postgresql.conf does not load with the error message:
FATAL: invalid value for parameter "lc_messages": "EN"

That's because "en" isn't a legal locale spec. "en_US", for instance,
would be valid. (At least on most platforms ... maybe Windoze is out in
left field?)

regards, tom lane

However, postgresql.conf does not load with the error message:
FATAL: invalid value for parameter "lc_messages": "EN"

That's because "en" isn't a legal locale spec. "en_US", for instance,
would be valid. (At least on most platforms ... maybe Windoze
is out in
left field?)

Tried that too. Tried a whole lot of combinations of both unix style
(sv_SE, sv, se, se_sv for example) and windows style (Swedish.Sweden,
Sweden.Swedish, Swedish.Sweden.1252, etc etc). *it never works*. It
*does work* if I set it as an environment variable.

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

Tried that too. Tried a whole lot of combinations of both unix style
(sv_SE, sv, se, se_sv for example) and windows style (Swedish.Sweden,
Sweden.Swedish, Swedish.Sweden.1252, etc etc). *it never works*. It
*does work* if I set it as an environment variable.

I tried setting a breakpoint at setlocale() and tracing through
postmaster startup. I now realize that we have a rather fundamental
problem, which is that the startup sequence is essentially:

setlocale(LC_COLLATE, "");
setlocale(LC_CTYPE, "");

// other uninteresting setlocales...

setlocale(LC_MESSAGES, value_from_config_file);
// and ditto for the other LC values in the config file

setlocale(LC_COLLATE, value_from_pg_control);
setlocale(LC_CTYPE, value_from_pg_control);

That is, postgresql.conf is read before we have located and read
pg_control. Therefore, if the LC_CTYPE value specified in pg_control
is different from whatever happens to be implied by the postmaster's
current environment, it is entirely possible for the set of allowed
LC_MESSAGES values during the initial config file read to be different
from the set that would be legal later. I am supposing here that
setlocale() may reject LC_MESSAGES values that imply a character set
different from that implied by LC_CTYPE.

Magnus, can you try a quick standalone test program to see if Windows'
setlocale seems to act that way? The FAQ you pointed at implies that
GNU gettext has some issues in this area, but it doesn't say outright
that the setlocale function itself fails.

If this is the problem then it might explain the various past complaints
we've gotten about being unable to set LC_MESSAGES on some operating
systems (OS X at least). My thoughts about fixing it are leaning
towards postponing processing of the LC_xxx config values until after
we've read pg_control, but that seems like a mess :-(

regards, tom lane

Magnus, can you try a quick standalone test program to see if Windows'
setlocale seems to act that way? The FAQ you pointed at implies that
GNU gettext has some issues in this area, but it doesn't say outright
that the setlocale function itself fails.

Yup, still doesn't seem to work. Assuming this is what you wanted me to
test ;-) Test program attached, results below. It returns NULL for
whatever I try with LC_MESSAGES. And yes, Windows has weird names for
locales :-) See
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclib/h
tml/_crt_language_strings.asp.

Or do I need to add some kind of initialization of the gettext library?

//Magnus

$ ./localetest.exe
LC_CTYPE -> NULL: C
LC_CTYPE -> C: C
LC_CTYPE -> en_US: (null)
LC_CTYPE -> english-us: English_United States.1252
LC_MESSAGES -> NULL: (null)
LC_MESSAGES -> C: (null)
LC_MESSAGES -> en_US: (null)
LC_MESSAGES -> english-us: (null)

"Magnus Hagander" <mha@sollentuna.net> writes:

Test program attached, results below. It returns NULL for
whatever I try with LC_MESSAGES.

It looks like LC_MESSAGES just plain does not work on Windows. I did
some googling and found some pages suggesting this, for instance

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclib/html/_crt_setlocale.2c_._wsetlocale.asp

The patch you suggest looks remarkably ugly; in particular it is
generally unsafe to pass a local variable to putenv. Perhaps that
does not matter on Windows but I wonder whether the putenv part couldn't
just be dropped. And why bother with ZeroMemory?

regards, tom lane

Test program attached, results below. It returns NULL for
whatever I try with LC_MESSAGES.

It looks like LC_MESSAGES just plain does not work on Windows. I did
some googling and found some pages suggesting this, for instance

http://msdn.microsoft.com/library/default.asp?url=/library/en-u
s/vclib/html/_crt_setlocale.2c_._wsetlocale.asp

Yes, that's exactly the link I had in my original mail.

The patch you suggest looks remarkably ugly; in particular it is
generally unsafe to pass a local variable to putenv. Perhaps that
does not matter on Windows but I wonder whether the putenv part

couldn't

just be dropped.

I got that from the gettext FAQ.
The problem is that MSVCRT caches the environment, and that's what you
access using getenv() and putenv(). And this is what gettext uses
internally. The API call is to change the actual process environment.
That's why you need both.

As for the dangers of passing a local variable - there is nothing abot
that in the putenv documentation on MSDN. So I would assume it's safe on
windows.

And why bother with ZeroMemory?

Oops. that's a leftover from some previous hacking.

//Magnus

"Magnus Hagander" <mha@sollentuna.net> writes:

Attached is a patch with adds a environment variable based version of
locale_messages_assign(). It's not a pretty solution, but I think it's
probably necessary.

Applied with minor cleanup.

I'm still concerned about the order-of-operations issue, but some
desultory experimentation on Linux (Fedora Core 3) did not find any
cases where setlocale(LC_MESSAGES) would fail because of a contradictory
LC_CTYPE setting, so maybe that theory is all wet. Still it seems there
is *something* fishy going on, given the number of reports we've seen.

regards, tom lane