ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8"

On 5 Jun 2010, at 4:44, Wang, Mary Y wrote:

Hi,

I'm getting this error from postgres " ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8" " and from a dump file when I tried to use psql command to restore the dump.

I have SET client_encoding = 'win1252' in the dump file.

Any ideas?

Apparently Win1252 is the not the right encoding to use for your data, as 0x90 is undefined in that encoding. See http://dbaspot.com/forums/ingres-database/414795-re-info-ingres-fw-how-insert-control-charactersintovarcharcolumn.html for someone with the same problem (on Ingres) and links to two different code pages listing the characters therein.

Alban Hertroys

--
If you can't see the forest for the trees,
cut the trees and you'll see there is no forest.

!DSPAM:737,4c0a0c3a10151431365867!

Thanks Alban, but I'm still confused.
I just read the PostgreSQL manual, and got the impression that UTF8 would take care of ALL character sets, so I put SET client_encoding = 'UTF8' in the dump file, and reload it again, then I got even more errors like:
psql:/tmp/060410finalsr15dump.txt:31430: ERROR: invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:31795: ERROR: invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:31826: ERROR: invalid byte sequence for encoding "UTF8": 0x85
psql:/tmp/060410finalsr15dump.txt:31827: ERROR: invalid byte sequence for encoding "UTF8": 0x92
psql:/tmp/060410finalsr15dump.txt:31874: ERROR: invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:32111: ERROR: invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:32112: ERROR: invalid byte sequence for encoding "UTF8": 0x92

What is the client encoding that I should set to so that it would take care all character sets? http://www.postgresql.org/docs/8.3/interactive/multibyte.html

Any suggestions?
Thanks
Mary

-----Original Message-----
From: Alban Hertroys [mailto:dalroi@solfertje.student.utwente.nl]
Sent: Saturday, June 05, 2010 1:35 AM
To: Wang, Mary Y
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8"

On 5 Jun 2010, at 4:44, Wang, Mary Y wrote:

Hi,

I'm getting this error from postgres " ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8" " and from a dump file when I tried to use psql command to restore the dump.

I have SET client_encoding = 'win1252' in the dump file.

Any ideas?

Apparently Win1252 is the not the right encoding to use for your data, as 0x90 is undefined in that encoding. See http://dbaspot.com/forums/ingres-database/414795-re-info-ingres-fw-how-insert-control-charactersintovarcharcolumn.html for someone with the same problem (on Ingres) and links to two different code pages listing the characters therein.

Alban Hertroys

--
If you can't see the forest for the trees, cut the trees and you'll see there is no forest.

!DSPAM:1099,4c0a0c3410156830940052!

On 5 Jun 2010, at 16:07, Wang, Mary Y wrote:

Thanks Alban, but I'm still confused.
I just read the PostgreSQL manual, and got the impression that UTF8 would take care of ALL character sets, so I put SET client_encoding = 'UTF8' in the dump file, and reload it again, then I got even more errors like:

Ah no, that won't work if your data isn't encoded according to UTF-8.

What is the client encoding that I should set to so that it would take care all character sets? http://www.postgresql.org/docs/8.3/interactive/multibyte.html

We can't know that. It's your client application(s) that put the data into your database, it depends on their encoding(s). That's the encoding you need.

If no encoding was specified you probably ended up with a mixed bag of encodings. AFAIK the only way out of that is to check everything by hand - a computer won't know what the data means, so isn't likely to be able to figure out what encoding was used.

If you don't care, set your database to use SQL_ASCII and it will just store the literal bytes as they are, without trying to interpret their encodings. If you already have a mixed bag of encodings, that's the easiest solution to get back to a working system - be aware though that you probably were already having issues with displaying some data correctly.

-----Original Message-----
From: Alban Hertroys [mailto:dalroi@solfertje.student.utwente.nl]
Sent: Saturday, June 05, 2010 1:35 AM
To: Wang, Mary Y
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8"

On 5 Jun 2010, at 4:44, Wang, Mary Y wrote:

Hi,

I'm getting this error from postgres " ERROR: character 0x90 of encoding "WIN1252" has no equivalent in "UTF8" " and from a dump file when I tried to use psql command to restore the dump.

I have SET client_encoding = 'win1252' in the dump file.

Any ideas?

Apparently Win1252 is the not the right encoding to use for your data, as 0x90 is undefined in that encoding. See http://dbaspot.com/forums/ingres-database/414795-re-info-ingres-fw-how-insert-control-charactersintovarcharcolumn.html for someone with the same problem (on Ingres) and links to two different code pages listing the characters therein.

Alban Hertroys

--
If you can't see the forest for the trees, cut the trees and you'll see there is no forest.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Alban Hertroys

--
Screwing up is an excellent way to attach something to the ceiling.

!DSPAM:737,4c0a80a710151031451958!

On Saturday 05 June 2010 7:07:35 am Wang, Mary Y wrote:

Thanks Alban, but I'm still confused.
I just read the PostgreSQL manual, and got the impression that UTF8 would
take care of ALL character sets, so I put SET client_encoding = 'UTF8' in
the dump file, and reload it again, then I got even more errors like:
psql:/tmp/060410finalsr15dump.txt:31430: ERROR: invalid byte sequence for
encoding "UTF8": 0x93 psql:/tmp/060410finalsr15dump.txt:31795: ERROR:
invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:31826: ERROR: invalid byte sequence for
encoding "UTF8": 0x85 psql:/tmp/060410finalsr15dump.txt:31827: ERROR:
invalid byte sequence for encoding "UTF8": 0x92
psql:/tmp/060410finalsr15dump.txt:31874: ERROR: invalid byte sequence for
encoding "UTF8": 0x93 psql:/tmp/060410finalsr15dump.txt:32111: ERROR:
invalid byte sequence for encoding "UTF8": 0x93
psql:/tmp/060410finalsr15dump.txt:32112: ERROR: invalid byte sequence for
encoding "UTF8": 0x92

What is the client encoding that I should set to so that it would take care
all character sets?
http://www.postgresql.org/docs/8.3/interactive/multibyte.html

Any suggestions?
Thanks
Mary

I am afraid it does not work that way. You have to think translation tables. In
order for the process to work Postgres needs to know the correct encoding it is
receiving in order to make the translation to UTF8. Much the same problem as
giving someone who knows how to translate French to English, a document that
contains a mixture of words in different languages. If they don't know what the
words are they cannot be translated.

--
Adrian Klaver
adrian.klaver@gmail.com

2010/6/4 Wang, Mary Y <mary.y.wang@boeing.com>:

Hi,

I'm getting this error from postgres " ERROR:  character 0x90 of encoding "WIN1252" has no equivalent in "UTF8"  " and from a dump file when I tried to use psql command to restore the dump.

I have SET client_encoding = 'win1252' in the dump file.

Any ideas?

0x90 isn't a valid WIN-1252 character.
http://msdn.microsoft.com/en-us/goglobal/cc305145.aspx

Osvaldo