BUG #4186: set lc_messages does not work

Thomas H wrote:

the patch discussed here [1] that supposedly made the win32 msvc builds use
lc_locale properly has flaws.

I think you misunderstood the feature [1]http://archives.postgresql.org/pgsql-patches/2008-04/msg00451.php added recently. This new
feature doesn't rely on lc_messages to localize the day and month names;
it uses lc_time. Look at the archives for reference.

[1]: http://archives.postgresql.org/pgsql-patches/2008-04/msg00451.php

--
Euler Taveira de Oliveira
http://www.timbira.com/

euler taveira de olivieira wrote:

the patch discussed here [1] that supposedly made the win32 msvc
builds use
lc_locale properly has flaws.

I think you misunderstood the feature [1] added recently. This new

actually no. the problem is as i intended to point out with the system
generated error messages. their language is supposedly controlled by
LC_MESSAGES (as was the time/date output, according to the first
february patch submission).

please observe the (previously already submitted) test queries. i've
removed the date/time testqueries to no further distract from the
problem. the bogus query "select x;" always results in a german error
messages no matter what LC_MESSAGES is set:

endor=# set lc_messages to 'sv_SE';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'de_DE';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'English_United_States';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;
^
endor=#
endor=# set lc_messages to 'fr';
SET
endor=# select x;
FEHLER: Spalte »x« existiert nicht
ZEILE 1: select x;

setting LC_MESSAGES in postgres.conf doesn't change anything either. and
the comment there says explicitly that LC_MESSAGES is used for system
errors:

lc_messages = 'English_United_States' # locale for system error message
# strings

nevertheless, everything is outputted in german now, errors as well as
logs. which seems to hint at 8.3.1 ignoring the LC_MESSAGES and always
using the os' locale.

regards,
thomas

Thomas H. wrote:

please observe the (previously already submitted) test queries. i've
removed the date/time testqueries to no further distract from the
problem. the bogus query "select x;" always results in a german error
messages no matter what LC_MESSAGES is set:

OK, that's another problem. AFAIK, that's a known problem because
Windows doesn't have LC_MESSAGES. The above comment (pg_locale.c)
suggests that there is no verification for the locale that is been set.
A possible solution is to use IsValidLocaleName() [1]http://msdn.microsoft.com/en-us/library/ms776379(VS.85).aspx or
LocaleNameToLCID() [2]http://msdn.microsoft.com/en-us/library/ms776388(VS.85).aspx but it seems that they're only available for
Vista. :( Maybe we could emulate one of these functions with a mapping
table [3]http://msdn.microsoft.com/en-us/library/ms776260.aspx. [searching ...] It seems there are problems with LCIDs; they
don't describe the locales acurately. pgwin hackers?
Could you try to use one of the locale names described in [4]?

/*
* LC_MESSAGES category does not exist everywhere, but accept it anyway
*
* On Windows, we can't even check the value, so the non-doit case is a
* no-op
*/

[1]: http://msdn.microsoft.com/en-us/library/ms776379(VS.85).aspx
[2]: http://msdn.microsoft.com/en-us/library/ms776388(VS.85).aspx
[3]: http://msdn.microsoft.com/en-us/library/ms776260.aspx

--
Euler Taveira de Oliveira
http://www.timbira.com/

Euler Taveira de Oliveira wrote:

please observe the (previously already submitted) test queries. i've
removed the date/time testqueries to no further distract from the
problem. the bogus query "select x;" always results in a german error
messages no matter what LC_MESSAGES is set:

OK, that's another problem. AFAIK, that's a known problem because
Windows doesn't have LC_MESSAGES. The above comment (pg_locale.c)
suggests that there is no verification for the locale that is been set.
A possible solution is to use IsValidLocaleName() [1] or
LocaleNameToLCID() [2] but it seems that they're only available for
Vista. :( Maybe we could emulate one of these functions with a mapping
table [3]. [searching ...] It seems there are problems with LCIDs; they
don't describe the locales acurately. pgwin hackers?

how does LC_MESSAGES differ from for example LC_TIME? in LC_TIME, the
checking of the specified locale seems to work:

endor=# set LC_MESSAGES = 'en-US';
SET
endor=# select x;
FEHLER: Spalte �x� existiert nicht
ZEILE 1: select x;
^
endor=# set LC_TIME = 'en-US';
FEHLER: ung�ltiger Wert f�r Parameter �lc_time�: �en-US�
endor=# set LC_TIME = 'en';
FEHLER: ung�ltiger Wert f�r Parameter �lc_time�: �en�
endor=# set LC_TIME = 'English';
SET

maybe one could as a workaround just use the lc_time locale checks for
lc_messages on windows systems? or at least match against the internal
pgsql supported translations. i don't mind having to specify "en"
instead of "English" if that gets me english error messages ;)

Could you try to use one of the locale names described in [4]?

i take it you meant link [3]. i've tried 'en-US' and others, same
problem, errors in german (excerpt above).

[1] http://msdn.microsoft.com/en-us/library/ms776379(VS.85).aspx
[2] http://msdn.microsoft.com/en-us/library/ms776388(VS.85).aspx
[3] http://msdn.microsoft.com/en-us/library/ms776260.aspx

regards,
thomas

"Thomas H" <me@alternize.com> writes:

Bug reference: 4186
Logged by: Thomas H
Email address: me@alternize.com
PostgreSQL version: 8.3.1
Operating system: Windows 2003
Description: set lc_messages does not work
Details:

the patch discussed here [1] that supposedly made the win32 msvc builds use
lc_locale properly has flaws.

I think a large part of the confusion that's been evidenced in this
thread is because you are submitting a "bug report" about a patch that is
not in fact in 8.3.1 (and is unlikely ever to appear in 8.3.x at all).
It's unclear what you are really testing: stock 8.3.1? 8.3.1 plus that
patch? CVS HEAD plus the patch?

regards, tom lane

From: Tom Lane <tgl@sss.pgh.pa.us>

the patch discussed here [1] that supposedly made the win32 msvc builds use
lc_locale properly has flaws.

I think a large part of the confusion that's been evidenced in this
thread is because you are submitting a "bug report" about a patch that is
not in fact in 8.3.1 (and is unlikely ever to appear in 8.3.x at all).
It's unclear what you are really testing: stock 8.3.1? 8.3.1 plus that
patch? CVS HEAD plus the patch?

it's the 8.3.1 win32 binary coming from postgresql.org i'm using.

i was digging in the mailing list archives for some answers and found
the conversation about the patch which i believed to be included in
8.3.1. sorry for the mess :|

nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
on the windows boxes, while it works on the non-win32 systems. all i
really would like is to get english system messages back on our
non-english win32 servers - like they were pre 8.3.1.

the main reason for this being: the german error messages have non-ascii
special characters in the error text ('�' and '�') which mess with some
of our applications (whereas english messages use '"').

regards,
thomas

"Thomas H." <me@alternize.com> writes:

nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
on the windows boxes, while it works on the non-win32 systems. all i
really would like is to get english system messages back on our
non-english win32 servers - like they were pre 8.3.1.

So far as I can tell, the backend's handling of LC_MESSAGES hasn't
changed at all between 8.1.2 and 8.3.1, so if it used to work for you
then there's been some other relevant change. Any idea what?

regards, tom lane

From: Tom Lane <tgl@sss.pgh.pa.us>

"Thomas H." <me@alternize.com> writes:

nevertheless the problem/bug exists: changing LC_MESSAGES has no effect
on the windows boxes, while it works on the non-win32 systems. all i
really would like is to get english system messages back on our
non-english win32 servers - like they were pre 8.3.1.

So far as I can tell, the backend's handling of LC_MESSAGES hasn't
changed at all between 8.1.2 and 8.3.1, so if it used to work for you
then there's been some other relevant change. Any idea what?

well... i'm not saying it worked in earlier versions. but prior to 8.3.1
(i've tested 8.3.0 and 8.2.3), the error messages were in english no
matter what the config file or the os locale says.

now, with 8.3.1, error & system messages are always in the os' locale,
and thus the bugreport. there is currently no way to set the pg system
messages' language, as the LC_MESSAGES setting seems to be defunct on
win32 systems.

as a small "proof", i installed the win32 8.3.0 from postgresql.org as a
fresh install and changed the postgresl.conf' lc_messages value to
'English_UnitedStates'. here's the psql output:

8.3.0:
-----------------------------------
postgres=# show lc_messages;
lc_messages
--------------------
English_UnitedStates
(1 row)

postgres=# select x;
ERROR: column "x" does not exist
LINE 1: select x;
^
postgres=# set lc_messages='French';
SET
postgres=# select x;
ERROR: column "x" does not exist
LINE 1: select x;
^
postgres=#
-----------------------------------

--> all system messages are in english.

after *upgrading* to 8.3.1 (again, using the official binaries), the
output looks like this:

8.3.1:
-----------------------------------
postgres=# show lc_messages;
lc_messages
----------------------
English_UnitedStates
(1 Zeile)

postgres=# select x;
FEHLER: Spalte �x� existiert nicht
ZEILE 1: select x;
^
postgres=# set lc_messages='French';
SET
postgres=# select x;
FEHLER: Spalte �x� existiert nicht
ZEILE 1: select x;
^
postgres=# show lc_messages;
lc_messages
-------------
French
(1 Zeile)

postgres=#

-----------------------------------

--> all system messages are in german (the os' locale)

so clearly between 8.3.0 and 8.3.1, something must have changed. but the
only patch that concerned win32 msvc/locale is the one you said wasn't
even included...

what i noticed: if i delete the folder share/locale/de/ the system
messages are back to english - but that can't be THE solution, can it? :)

regards,
thomas

From: Thomas H. <me@alternize.com>

what i noticed: if i delete the folder share/locale/de/ the system
messages are back to english - but that can't be THE solution, can it? :)

well, it actually was the solution, at least to the weird part of the
problem:

there are two versions of win32 binaries available on postgres.org.
there's the installer msi version, and there is the installerless zip
version.

the installer version does NOT install the folder "share/locale/"! so
when using the msi installer, postgres has no translations at all - thus
the fallback to english.

the zipped version contains the "share/locale/" folder. for installing
8.3.0, i've been using the msi installer version. for upgrading to
8.3.1, i've been unpacking the files from zip version into the
postgresql directory...

i've just verified that the 8.3.1 msi version provided on postgres.org
also does NOT contain the "locale" folder & files. should i report this
as a separate bug/problem?

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present, pg
always picks the os' language and ignores the lc_message value.

- thomas

On Thu, May 29, 2008 at 2:05 AM, Thomas H. <me@alternize.com> wrote:

From: Thomas H. <me@alternize.com>

i've just verified that the 8.3.1 msi version provided on postgres.org also
does NOT contain the "locale" folder & files. should i report this as a
separate bug/problem?

How exactly did you do that? My installation certainly has it. If
memory serves, it's not installed by default (you have to select it in
the feature list), but it's there alright.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

Thomas H. wrote:

From: Thomas H. <me@alternize.com>

what i noticed: if i delete the folder share/locale/de/ the system
messages are back to english - but that can't be THE solution, can
it? :)

well, it actually was the solution, at least to the weird part of the
problem:

there are two versions of win32 binaries available on postgres.org.
there's the installer msi version, and there is the installerless zip
version.

the installer version does NOT install the folder "share/locale/"! so
when using the msi installer, postgres has no translations at all -
thus the fallback to english.

Are you sure you didn't just skip enabling National Language Support?
It's not included by default, but should be available.

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present,
pg always picks the os' language and ignores the lc_message value.

This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?

//Magnus

From: Dave Page <dpage@pgadmin.org>

On Thu, May 29, 2008 at 2:05 AM, Thomas H. <me@alternize.com> wrote:

From: Thomas H. <me@alternize.com>
i've just verified that the 8.3.1 msi version provided on postgres.org also
does NOT contain the "locale" folder & files. should i report this as a
separate bug/problem?

How exactly did you do that? My installation certainly has it. If
memory serves, it's not installed by default (you have to select it in
the feature list), but it's there alright.

hmm by just clicking through the standard settings. i've seen now that
the "national language support" is set to do not install by default. so
it is a feature, not a bug, sorry.

i was under the obviously wrong impression a zip-file upgrade would be
the same as an msi-upgrade.

- thomas

On Thu, May 29, 2008 at 12:23 PM, Thomas H. <me@alternize.com> wrote:

i was under the obviously wrong impression a zip-file upgrade would be the
same as an msi-upgrade.

Eeep - don't do that!! That is a *very* good way to annoy the Windows
Installer and trick it into thinking things need to be repaired.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

Magnus Hagander <magnus@hagander.net> writes:

Thomas H. wrote:

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present,
pg always picks the os' language and ignores the lc_message value.

This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?

Maybe we were using a different build of gettext in the previous
releases, one that didn't look at the same info as the current code?

Anyway the patch mentioned at the start of the thread
http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php
purports to fix this. It doesn't seem to have gotten reviewed
though.

regards, tom lane

Tom Lane wrote:

Magnus Hagander <magnus@hagander.net> writes:

Thomas H. wrote:

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present,
pg always picks the os' language and ignores the lc_message value.

This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?

Maybe we were using a different build of gettext in the previous
releases, one that didn't look at the same info as the current code?

Anyway the patch mentioned at the start of the thread
http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php
purports to fix this. It doesn't seem to have gotten reviewed
though.

Agreed. Magnus, someone, can we get feedback on the patch at this URL?

http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

Tom Lane wrote:

Magnus Hagander <magnus@hagander.net> writes:

Thomas H. wrote:

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present,
pg always picks the os' language and ignores the lc_message value.

This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?

Maybe we were using a different build of gettext in the previous
releases, one that didn't look at the same info as the current code?

Anyway the patch mentioned at the start of the thread
http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php
purports to fix this. It doesn't seem to have gotten reviewed
though.

Agreed. Magnus, someone, can we get feedback on the patch at this URL?

http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php

IIRC, there was further work to be done on the patch before it was to be
applied, and we held off the review until then.

Gevik - can you comment on this? Where are we, what needs ot be done still?

//Magnus

Hi Magnus.

I have tried with Inoue-san, investigation of this problem, and adjustment.
http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/pg8.3.3-to_char_gettext_format.png
Native-strftime was proposed by Tom-san. It corrects(LC_TIME) from 8.3.3.(LC_MESSAGES)
http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/
http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/pg_locale_patch
It has not resulted in the still final conclusion.
However, Possibly this may be the optimal one.
http://winpg.jp/~saito/pg_work/LC_MESSAGE_CHECK/LC_TIME_PATCH/wcsftimeck.c
I am creating the proposal of patch with Inoue-san.

Regards,
Hiroshi Saito

----- Original Message -----
From: "Magnus Hagander" <magnus@hagander.net>

My apologies on this late reply.
The way LC_MESSAGES is handled on windows is much less efficient and faulty.
While ago I started with a patch to fix some of the issues I encountered on
windows and LC_MESSAGES. But I stopped working on that patch because this
problem needed to be fixed on many other places. In Windows, handling
LC_MESSAGES will not work the same way as *nix systems, forcing us to make
ugly workarounds. (as I actually was doing with my patch)

To my opinion, unless we think of a coherent solution for handling
LC_MESSAGES/locale for both *nix and win32 platforms, fixing lc_messages and
locale issues would break more than fixing it.

BTW: The gettext library we are using on win32 is a very old one.

Regards,
Gevik.

Tom Lane wrote:

Magnus Hagander <magnus@hagander.net> writes:

Thomas H. wrote:

so at least that explains the "changed" behaviour. nevertheless,
LC_MESSAGES seems to be defunct - with the "locale" folder present,
pg always picks the os' language and ignores the lc_message value.

This looks like I can reproduce though, at least on cvs head. Did this
work for you in previous versions?

Maybe we were using a different build of gettext in the previous
releases, one that didn't look at the same info as the current code?

Anyway the patch mentioned at the start of the thread
http://archives.postgresql.org/pgsql-patches/2008-02/msg00038.php
purports to fix this. It doesn't seem to have gotten reviewed
though.

Where are we on this?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +