latin1 unicode conversion errors

Started by Kris Jurkaabout 20 years ago3 messageshackers
Jump to latest
#1Kris Jurka
books@ejurka.com

Why is latin1 special in its conversion from unconvertible unicode data?
Other latin character sets add a warning, but latin1 errors out.

jurka=# create database utf8 with encoding ='utf8';
CREATE DATABASE
jurka=# \c utf8
You are now connected to database "utf8".
utf8=# create table t(a text);
CREATE TABLE
utf8=# insert into t values ('\346\231\243');
INSERT 0 1
utf8=# set client_encoding = 'latin2';
SET
utf8=# select * from t;
WARNING: ignoring unconvertible UTF-8 character 0xe699a3
a
---

(1 row)

utf8=# set client_encoding = 'latin1';
SET
utf8=# select * from t;
ERROR: could not convert UTF8 character 0x00e6 to ISO8859-1

Kris Jurka

#2Bruce Momjian
bruce@momjian.us
In reply to: Kris Jurka (#1)
Re: latin1 unicode conversion errors

My guess is that it was coded by someone different and needs to be made
consistent.

---------------------------------------------------------------------------

Kris Jurka wrote:

Why is latin1 special in its conversion from unconvertible unicode data?
Other latin character sets add a warning, but latin1 errors out.

jurka=# create database utf8 with encoding ='utf8';
CREATE DATABASE
jurka=# \c utf8
You are now connected to database "utf8".
utf8=# create table t(a text);
CREATE TABLE
utf8=# insert into t values ('\346\231\243');
INSERT 0 1
utf8=# set client_encoding = 'latin2';
SET
utf8=# select * from t;
WARNING: ignoring unconvertible UTF-8 character 0xe699a3
a
---

(1 row)

utf8=# set client_encoding = 'latin1';
SET
utf8=# select * from t;
ERROR: could not convert UTF8 character 0x00e6 to ISO8859-1

Kris Jurka

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
#3Bruce Momjian
bruce@momjian.us
In reply to: Kris Jurka (#1)
Re: latin1 unicode conversion errors

OK, yea, it is inconsistent. I changed it do throw a warning instead.
Only patched to 8.2 because it is a behavior change.

---------------------------------------------------------------------------

Kris Jurka wrote:

Why is latin1 special in its conversion from unconvertible unicode data?
Other latin character sets add a warning, but latin1 errors out.

jurka=# create database utf8 with encoding ='utf8';
CREATE DATABASE
jurka=# \c utf8
You are now connected to database "utf8".
utf8=# create table t(a text);
CREATE TABLE
utf8=# insert into t values ('\346\231\243');
INSERT 0 1
utf8=# set client_encoding = 'latin2';
SET
utf8=# select * from t;
WARNING: ignoring unconvertible UTF-8 character 0xe699a3
a
---

(1 row)

utf8=# set client_encoding = 'latin1';
SET
utf8=# select * from t;
ERROR: could not convert UTF8 character 0x00e6 to ISO8859-1

Kris Jurka

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Attachments:

/bjm/difftext/plainDownload+6-6