postgres.app OS X psql character encoding (utf-8) problem
Hi everyone,
I am running Postgres93.app (v 9.3.1). I'm new to Postgres, but experienced with MySQL and *nix. I'm having a problem with the command line client "psql" while connecting to postgress.app running on the localhost on OS X. It's behaving as if somewhere a character encoding is set incorrectly, but as far as I can tell, 'UTF-8' or 'UTF8' is correctly set everywhere.
If I enter a unicode character in the psql cli, such as:
user=# select 'ö';
But before hitting enter, use the keyboard "left" button to move the cursor across all the way to the left edge, and then back all the way to the right, the output get distorted and looks like this:
user=#select 'ö';;
If I do hit enter - the command works fine, so the problem is on the display/client side.
Here's why I think all my encodings are set correctly:
user@hostname:~ $ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
user@hostname:~ $ psql
psql (9.3.1)
Type "help" for help.
user=# \encoding
UTF8
user=# \l user
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
--------+--------+----------+-------------+-------------+-------------------
user | user | UTF8 | en_US.UTF-8 | en_US.UTF-8 |
Also if, from my OS X terminal, I ssh into a linux box and run 'psql', the problem does not reproduce. So I'm fairly certain my terminal is not the problem.
Can anyone reproduce the problem, or have any advice about something I've missed?
Thanks,
Chris
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Chris Hiestand <chiestand@salk.edu> writes:
If I enter a unicode character in the psql cli, such as:
user=# select '�';
But before hitting enter, use the keyboard "left" button to move the cursor across all the way to the left edge, and then back all the way to the right, the output get distorted and looks like this:
user=#select '�';;
What this sounds like is that the readline or libedit library doesn't
understand multibyte characters properly. psql itself doesn't have
anything to do with the display of un-entered lines, but relies on
one of those libraries to manage input editing.
The default situation on OS X is generally that psql gets linked against
the Apple-supplied libedit, which goes so far as to masquerade as
readline. Check "otool -L /path/to/psql"; if you see a reference to
/usr/lib/libedit.3.dylib, or to /usr/lib/libreadline.dylib
(which is really just a symlink to the former), then that's what you've
got. While I'm generally a fan and user of Apple stuff, their version
of libedit is just abysmal; we've seen random crashes, complete failure
of tab completion, and other bugs in successive OS X releases. I also
find specific references to multibyte input being busted in other
distros' versions of libedit, eg
/messages/by-id/4D5B2C5A.8090506@catalyst.net.nz
so it may not be all Apple's fault; but they definitely have a track
record of shipping broken versions of libedit.
I'd strongly recommend installing the GNU readline library and rebuilding
psql against that.
regards, tom lane
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
On Jan 9, 2014, at 1:09 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
What this sounds like is that the readline or libedit library doesn't
understand multibyte characters properly. psql itself doesn't have
anything to do with the display of un-entered lines, but relies on
one of those libraries to manage input editing.The default situation on OS X is generally that psql gets linked against
the Apple-supplied libedit, which goes so far as to masquerade as
readline. Check "otool -L /path/to/psql"; if you see a reference to
/usr/lib/libedit.3.dylib, or to /usr/lib/libreadline.dylib
(which is really just a symlink to the former), then that's what you've
got. While I'm generally a fan and user of Apple stuff, their version
of libedit is just abysmal; we've seen random crashes, complete failure
of tab completion, and other bugs in successive OS X releases. I also
find specific references to multibyte input being busted in other
distros' versions of libedit, eg
/messages/by-id/4D5B2C5A.8090506@catalyst.net.nz
so it may not be all Apple's fault; but they definitely have a track
record of shipping broken versions of libedit.
Thanks Tom Lane. It's useful to know that one of these handles unentered text.
postgres93.app is supplied with its own version of libedit.
user@hostname:~ $ otool -L `which psql`
/Applications/Postgres93.app/Contents/MacOS/bin/psql:
/Applications/Postgres93.app/Contents/MacOS/lib/libpq.5.dylib (compatibility version 5.0.0, current version 5.6.0)
/Applications/Postgres93.app/Contents/MacOS/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/Applications/Postgres93.app/Contents/MacOS/lib/libedit.0.dylib (compatibility version 1.0.0, current version 1.45.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
Do you think this an indication that the app bundle may have been compiled with a bad version of libedit? I can file a bug report along those lines.
Thanks,
Chris
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
Chris Hiestand <chiestand@salk.edu> writes:
postgres93.app is supplied with its own version of libedit.
user@hostname:~ $ otool -L `which psql`
/Applications/Postgres93.app/Contents/MacOS/bin/psql:
/Applications/Postgres93.app/Contents/MacOS/lib/libpq.5.dylib (compatibility version 5.0.0, current version 5.6.0)
/Applications/Postgres93.app/Contents/MacOS/lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/Applications/Postgres93.app/Contents/MacOS/lib/libedit.0.dylib (compatibility version 1.0.0, current version 1.45.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)
Uh-huh. You need to complain to whoever packages Postgres93.app, then.
The impression I have about libedit is that it's necessary to grab a
snapshot of upstream on a good day ;-)
regards, tom lane
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general
On Jan 9, 2014, at 8:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Uh-huh. You need to complain to whoever packages Postgres93.app, then.
This is done: https://github.com/PostgresApp/PostgresApp/issues/165
Thanks Tom, I appreciate your time.
-Chris
--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general