COPY FROM encoding error

Started by Arnaud Lesauvageover 19 years ago2 messagesgeneral
Jump to latest
#1Arnaud Lesauvage
thewild@freesurf.fr

Hi list !

I am trying to migrate a database from MSSQL to PostgreSQL.
I created the table in PostgreSQL, and I exported the data
as CSV from MSSQL's Enterprise Manager.

In the "Export Data" Wizard, I chose the option to export as
UNICODE.

In PostgreSQL, I try to load the data using :

COPY mytable (col1, col2, col3)
FROM 'mytable.csv' CSV;

But I receive this error after some time :
ERROR: invalid byte sequence for encoding "UTF8": 0xff
�tat SQL :22021
Astuce : This error can also happen if the byte sequence
does not match the encoding expected by the server, which is
controlled by "client_encoding".
Contexte : COPY mytable, line 592680

I think that the encoding is OK because more than 500.000
lines are copied without problem, so there might be just one
problematic character here.

How can I solve this problem ?

Thanks a lot !
--
Arnaud

#2Jim Nasby
Jim.Nasby@BlueTreble.com
In reply to: Arnaud Lesauvage (#1)
Re: COPY FROM encoding error

On Nov 21, 2006, at 4:20 AM, Arnaud Lesauvage wrote:

ERROR: invalid byte sequence for encoding "UTF8": 0xff
État SQL :22021
Astuce : This error can also happen if the byte sequence does not
match the encoding expected by the server, which is controlled by
"client_encoding".
Contexte : COPY mytable, line 592680

I think that the encoding is OK because more than 500.000 lines are
copied without problem, so there might be just one problematic
character here.

How can I solve this problem ?

You need to fix the bad character. You can do this manually, or
search the archives for "UTF8 invalid iconv" for another solution.
--
Jim Nasby jim.nasby@enterprisedb.com
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)