NLS on MSVC strikes back!

Gevik Babakhani wrote:

I have compiled 8.3 (CVS HEAD) with NLS on a XP box. But it seems that NLS
support is broke.
Could someone please confirm this.

My test:

Gevik=#
Gevik=# set lc_messages TO 'de_DE.utf8';
SET
Gevik=# show lc_messages ;
lc_messages
-------------
de_DE.utf8
(1 row)

Gevik=# select to_char(now(), 'TMDay, DD TMMonth YYYY');
to_char
---------------------------
Tuesday, 12 February 2008
(1 row)

Should that not be lc_time you are setting? lc_messages is for, uh,
messages.

cheers

andrew

Should that not be lc_time you are setting? lc_messages is
for, uh, messages.

No. The same thing works on 8.2.6

Gevik Babakhani wrote:

Should that not be lc_time you are setting? lc_messages is
for, uh, messages.

No. The same thing works on 8.2.6

Well, that should be considered a bug, not a feature. Perhaps it was
fixed in 8.3.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Well, that should be considered a bug, not a feature.
Perhaps it was fixed in 8.3.

This is 8.3 I am testing with.

Gevik Babakhani wrote:

Well, that should be considered a bug, not a feature.
Perhaps it was fixed in 8.3.

This is 8.3 I am testing with.

Right, I know that. But I didn't see you trying lc_time as Andrew
suggested.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Gevik Babakhani wrote:

Well, that should be considered a bug, not a feature.
Perhaps it was fixed in 8.3.

This is 8.3 I am testing with.

You are missing Alvaro's point. He is saying that the behaviour you
relied on in 8.2.6 was a bug, and possibly you can no longer rely on the
buggy behaviour in 8.3.

cheers

andrew

Right, I know that. But I didn't see you trying lc_time as
Andrew suggested.

Did that too, but no luck :(

Gevik Babakhani wrote:

I have compiled 8.3 (CVS HEAD) with NLS on a XP box. But it seems that NLS
support is broke.
Could someone please confirm this.

What I can confirm is that lc_messages is supposed to work for to_char,
because it shows the localized output for me as I change lc_messages;
and it doesn't if I change lc_time.

In any case,

Gevik=# show lc_messages ;
lc_messages
----------------------------
English_United States.1252
(1 row)

Gevik=#
Gevik=# set lc_messages TO 'de_DE.utf8';
SET

I wonder if this is really valid. Shouldn't you be using a locale name
like 'German_Germany.65001' or some such? I thought Windows did not
recognize the de_DE.utf8 form of names. Perhaps it failed to raise an
error here? It does for me:

alvherre=# set lc_time to 'fr_CA.utf8';
SET
alvherre=# set lc_time to 'de_DE.utf8';
ERROR: valor no v�lido para el par�metro �lc_time�: �de_DE.utf8�

Suffice to say I don't have the de_DE locale installed.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera wrote:

Gevik Babakhani wrote:

I have compiled 8.3 (CVS HEAD) with NLS on a XP box. But it seems that NLS
support is broke.
Could someone please confirm this.

What I can confirm is that lc_messages is supposed to work for to_char,
because it shows the localized output for me as I change lc_messages;
and it doesn't if I change lc_time.

FSVO "supposed to".

The CVS history is somewhat murky.

On Nov 24 2006 this was committed dor src/backend/utils/adt/formatting.c:

r 1.114: Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

A few hours later it was supposedly reverted: r 1.115: Revert (too late
in beta): Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

Apparently this was never returned to (it should probably have been put
on the patches hold queue).

I can't see offhand what else might have caused the behaviour change.

cheers

andrew

Hi

--Document.--
Table 9-22.
TM prefix translation mode (print localized day and month names based on lc_messages)
--
http://winpg.jp/~saito/pg_work/NLS_TO_CHAR_JP.png
But, Although Japanese is out of condition....

Does the fundamental specification change?

Regards,
Hiroshi Saito

----- Original Message -----
From: "Alvaro Herrera" <alvherre@commandprompt.com>

Andrew Dunstan wrote:

Alvaro Herrera wrote:

Gevik Babakhani wrote:

I have compiled 8.3 (CVS HEAD) with NLS on a XP box. But it seems that NLS
support is broke.
Could someone please confirm this.

What I can confirm is that lc_messages is supposed to work for to_char,
because it shows the localized output for me as I change lc_messages;
and it doesn't if I change lc_time.

FSVO "supposed to".

The CVS history is somewhat murky.

On Nov 24 2006 this was committed dor src/backend/utils/adt/formatting.c:

r 1.114: Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

A few hours later it was supposedly reverted: r 1.115: Revert (too late
in beta): Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

Apparently this was never returned to (it should probably have been put
on the patches hold queue).

Added to TODO list:

o Use LC_TIME for localized weekday/month names, rather than
LC_MESSAGES

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Andrew Dunstan <andrew@dunslane.net> writes:

On Nov 24 2006 this was committed dor src/backend/utils/adt/formatting.c:

r 1.114: Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

A few hours later it was supposedly reverted: r 1.115: Revert (too late
in beta): Fix to_char() locale handling to honor LC_TIME, not LC_MESSAGES.

Apparently this was never returned to (it should probably have been put
on the patches hold queue).

to_char's month/day name localization is implemented with gettext() not
strftime(), which is why it depends on LC_MESSAGES not LC_TIME. I seem
to recall that we didn't like the side-effects of the patch you are
mentioning, and so it ended up being rejected outright.

regards, tom lane

Hiroshi Saito wrote:

Hi

--Document.--
Table 9-22.
TM prefix translation mode (print localized day and month names based on lc_messages)
--
http://winpg.jp/~saito/pg_work/NLS_TO_CHAR_JP.png
But, Although Japanese is out of condition....

Does the fundamental specification change?

Humm, I cannot read Japanese so as far as I can make, this is working fine?

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

to_char's month/day name localization is implemented with
gettext() not strftime(), which is why it depends on
LC_MESSAGES not LC_TIME. I seem to recall that we didn't
like the side-effects of the patch you are mentioning, and so
it ended up being rejected outright.

Correct. I have been looking for the cause of this problem and
these are my finding:

1. The platforms discussed and tested here are MS Windows XP and 2003 using
Microsoft VC++ 2005 and NOT MINGW.

2. As Tom described above, to_char is implemented with gettext which depends
on LC_MESSAGES. I guess this is okay.

3. Changing LC_MESSAGES is done in pg_locale.c:94:pg_perm_setlocale by
setting LC_MESSAGES environment variable. This works for MINGW (tested by
installing PG 8.2.6), but it does not work for MSVC++ due different locale
handling of gettext. Please see gettext sources:1087:localenames.c

4. Locale names are different in MS Windows. I created a C app to test
gettext on Windows.
setting LC_MESSAGES to Spanisg_Spain and German_Germany works but es_ES and
de_DE do not :(

SET LC_MESSAGES to '....' has no effect because:

A. gettext compiled/linked in MSVC looks for the locale of the current
thread and NOT the LC_MESSAGES,LANGIAGE,LANG... environment variables. See
gettext's sources....

B. Given current thread's locale Spanish_Spain, gettext fails to find a
directory called "Spanish_Spain" in share/locale. As result English names
are returned.

A possible solutions:

- Keep to_char/LC_MESSAGES handling and create a fix for B.
Because we do not want to change/maintain our own version of Gettext this
would also involve creating a different directory/name structure for
Windows. For example share\locale\de would be share\locale\German_Germany.

Any thoughts,

Regards,
Gevik Babakhani
------------------------------------------------
PostgreSQL NL http://www.postgresql.nl
TrueSoftware BV http://www.truesoftware.nl
------------------------------------------------

Gevik Babakhani wrote:

4. Locale names are different in MS Windows. I created a C app to test
gettext on Windows.
setting LC_MESSAGES to Spanisg_Spain and German_Germany works but es_ES and
de_DE do not :(

SET LC_MESSAGES to '....' has no effect because:

A. gettext compiled/linked in MSVC looks for the locale of the current
thread and NOT the LC_MESSAGES,LANGIAGE,LANG... environment variables. See
gettext's sources....

B. Given current thread's locale Spanish_Spain, gettext fails to find a
directory called "Spanish_Spain" in share/locale. As result English names
are returned.

What this really means is that locale support is completely broken in
the MSVC build, i.e. you cannot get localized strings at all (not just
to_char()). Is this correct? If so, this is a serious problem.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Feb 12, 2008 4:44 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote:

What this really means is that locale support is completely broken in
the MSVC build, i.e. you cannot get localized strings at all (not just
to_char()). Is this correct? If so, this is a serious problem.

Not judging by Hiroshi's screenshot. I don't speak Japanese either,
but it certainly didn't say 'tuesday' after he adjusted LC_MESSAGES.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Oracle-compatible database company

Dave Page wrote:

On Feb 12, 2008 4:44 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote:

What this really means is that locale support is completely broken in
the MSVC build, i.e. you cannot get localized strings at all (not just
to_char()). Is this correct? If so, this is a serious problem.

Not judging by Hiroshi's screenshot. I don't speak Japanese either,
but it certainly didn't say 'tuesday' after he adjusted LC_MESSAGES.

But it has the string "mingw" in the path, so I suspect it's not MSVC ...
Hiroshi-san, can you confirm?

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:

Gevik Babakhani wrote:

4. Locale names are different in MS Windows. I created a C app to test
gettext on Windows.
setting LC_MESSAGES to Spanisg_Spain and German_Germany works but es_ES and
de_DE do not :(

SET LC_MESSAGES to '....' has no effect because:

A. gettext compiled/linked in MSVC looks for the locale of the current
thread and NOT the LC_MESSAGES,LANGIAGE,LANG... environment variables. See
gettext's sources....

B. Given current thread's locale Spanish_Spain, gettext fails to find a
directory called "Spanish_Spain" in share/locale. As result English names
are returned.

What this really means is that locale support is completely broken in
the MSVC build, i.e. you cannot get localized strings at all (not just
to_char()). Is this correct? If so, this is a serious problem.

I've certainly managed to get output in Swedish more than once...
Annoying as hell because it's the default if you build with NLS and has
your English language Windows configured with Swedish time/date format :)

//Magnus

What this really means is that locale support is completely
broken in the MSVC build, i.e. you cannot get localized
strings at all (not just to_char()). Is this correct? If
so, this is a serious problem.

The way one could confirm this is by:

1. rename share/locale/de to share/locale/German_Germany
2. start a dosbox
3. before starting PG, set an env variable SET LANGUAGE=German_Germany in
that dosbox, which will force all child threads to have that locale.
3. start PG form that dosbox and SET LC_MESSAGES to "es_ES" or
"Spanish_Spain" or "es_ES.utf8" or anything else...; select
to_char(now(),'TMDay TMMonth YYYY');
4. The result will be in German and not Spanish as one expects.

If handling locale in to_char will remain using gettext then I can start
working on a patch to fix this.

Regards,
Gevik Babakhani
------------------------------------------------
PostgreSQL NL http://www.postgresql.nl
TrueSoftware BV http://www.truesoftware.nl
------------------------------------------------

Hi Alvaro-san.

Yes, However, It is not in good condition at the reason a message catalog still is not enough.
Then, I have tried these adjustments with the problem of a locale. Still, it is not much time.
But, My condition is not good so that I'm may be influenza. :-(

Regards,
Hiroshi Saito

----- Original Message -----
From: "Alvaro Herrera" <alvherre@commandprompt.com>

From: "Alvaro Herrera" <alvherre@commandprompt.com>

Dave Page wrote:

On Feb 12, 2008 4:44 PM, Alvaro Herrera <alvherre@commandprompt.com> wrote:

What this really means is that locale support is completely broken in
the MSVC build, i.e. you cannot get localized strings at all (not just
to_char()). Is this correct? If so, this is a serious problem.

Not judging by Hiroshi's screenshot. I don't speak Japanese either,
but it certainly didn't say 'tuesday' after he adjusted LC_MESSAGES.

But it has the string "mingw" in the path, so I suspect it's not MSVC ...
Hiroshi-san, can you confirm?

Yeah, However, I go to a hospital from now.... sorry.
I will check it, after returning.

Regards,
Hiroshi Saito

Hi.

Still, my health is not good...

First, Probably, as for the 8.3 release binary, NLS is offed.
Even if it arranges shar/locale, it is not used.

Next, NLS was confirmed by VCBUILD of CVS-HEAD.
A very interesting result can be seen here.
http://inet.winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/

It was expressed by LC_MESSAGE=C.
Anyway, in Japan, it is hard to use......

Regards,
Hiroshi Saito