JDBC connections to 9.1

Steve Singer <ssinger@ca.afilias.info> writes:

I'm getting JDBC exceptions when I try to connect to 9.1 (master) with
the postgresql-9.0-801.jdbc3.jar I don't have this issue with 9.0.

Hmm, what shows up in the postmaster log?

regards, tom lane

--On 18. April 2011 09:44:38 -0400 Tom Lane <tgl@sss.pgh.pa.us> wrote:

I'm getting JDBC exceptions when I try to connect to 9.1 (master) with
the postgresql-9.0-801.jdbc3.jar I don't have this issue with 9.0.

Hmm, what shows up in the postmaster log?

A quick check with an application here gives the following with JDBC
loglevel=2

16:09:47.910 (1) PostgreSQL 9.1devel JDBC4 (build 900)
16:09:47.914 (1) Trying to establish a protocol version 3 connection to
localhost:5438
16:09:47.930 (1) FE=> StartupPacket(user=bernd, database=mailstore,
client_encoding=UNICODE, DateStyle=ISO, extra_float_digits=2)
16:09:47.933 (1) <=BE AuthenticationOk
16:09:47.942 (1) <=BE ParameterStatus(application_name = )
16:09:47.942 (1) <=BE ParameterStatus(client_encoding = UTF8)
org.postgresql.util.PSQLException: Protocol error. Session setup failed.
at
org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:498)
at
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112)
at
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at
org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
at
org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
at
org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
at
org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:154)
at de.oopsware.mailstore.PGSQLMailstore.connect(Unknown Source)
at de.oopsware.mailstore.PGSQLMailstore.connect(Unknown Source)
at mailstore.main(Unknown Source)
SQLException: SQLState(08P01)
getConnection failed: org.postgresql.util.PSQLException: Protocol error.
Session setup failed.
org.postgresql.util.PSQLException: Protocol error. Session setup failed.

Hmm, seems it stumbles while reading client_encoding....

--
Thanks

Bernd

On 11-04-18 09:44 AM, Tom Lane wrote:

Steve Singer<ssinger@ca.afilias.info> writes:

I'm getting JDBC exceptions when I try to connect to 9.1 (master) with
the postgresql-9.0-801.jdbc3.jar I don't have this issue with 9.0.

Hmm, what shows up in the postmaster log?

regards, tom lane

LOG: unexpected EOF on client connection

On Mon, 2011-04-18 at 16:17 +0200, Bernd Helmle wrote:

Hmm, seems it stumbles while reading client_encoding....

This is probably similar to what I had a couple weeks ago. With today's
new minor releases, I get:

$ psql
psql: invalid connection option "client_encoding"

(I was getting another message before)

This is 9.1's psql connecting to 9.1's server. For some reason, psql
uses 9.0 client libs, so if I export LD_LIBRARY_PATH, everything works:

$ LD_LIBRARY_PATH=/usr/pgsql-9.1/lib psql
...

Could it be the same thing?
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz

--On 18. April 2011 16:17:57 +0200 Bernd Helmle <mailings@oopsware.de> wrote:

16:09:47.942 (1) <=BE ParameterStatus(client_encoding = UTF8)
org.postgresql.util.PSQLException: Protocol error. Session setup failed.
at
org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFa
ctoryImpl.java:498)

ConnectionFactoryImpl.readStartupMessages() has this:

else if (name.equals("client_encoding"))

{

if (!value.equals("UNICODE"))

throw new PSQLException(GT.tr("Protocol error. Session
setup failed."), PSQLState.PROTOCOL_VIOLATION);

pgStream.setEncoding(Encoding.getDatabaseEncoding("UNICODE"));

}

If i am reading it correct, it reads "UTF8" from the backend, while expecting
"UNICODE" only. Not sure what change has caused this, though. If i extend the
check to include "UTF8", everything seems to work.

--
Thanks

Bernd

Bernd Helmle <mailings@oopsware.de> writes:

If i am reading it correct, it reads "UTF8" from the backend, while
expecting "UNICODE" only. Not sure what change has caused this,
though.

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over. For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UnIcOdE
(1 row)

versus HEAD:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UTF8
(1 row)

I wasn't aware that JDBC would fail on that. It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

regards, tom lane

On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bernd Helmle <mailings@oopsware.de> writes:

If i am reading it correct, it reads "UTF8" from the backend, while
expecting "UNICODE" only. Not sure what change has caused this,
though.

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over.  For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UnIcOdE
(1 row)

versus HEAD:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UTF8
(1 row)

I wasn't aware that JDBC would fail on that.  It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

Ouch. I hate to revert that, since it seems like a clear improvement.
But I also hate to break JDBC. Ouch.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bernd Helmle <mailings@oopsware.de> writes:

If i am reading it correct, it reads "UTF8" from the backend, while
expecting "UNICODE" only. Not sure what change has caused this,
though.

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over.  For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UnIcOdE
(1 row)

versus HEAD:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UTF8
(1 row)

I wasn't aware that JDBC would fail on that.  It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

                       regards, tom lane

Older drivers will fail for sure. We can fix newer drivers, but if we
leave it we will see a slew of bug reports.

The reason the driver does this is to guarantee the client encoding is unicode.

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

Tom Lane <tgl@sss.pgh.pa.us> wrote:

I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the
client sent over. For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET

The whole area of character sets and encoding schemes is confusing
enough without accepting a character set name as an encoding scheme
specification. I'll bet that in five or ten years we'll be
accepting more than one encoding scheme for the Unicode character
set.

I wasn't aware that JDBC would fail on that. It's pretty annoying
that it does, but maybe we should grin and bear it, ie revert the
change to canonicalize the GUC's value?

Can we fix the JDBC driver rather than reverting this? Long run,
I'd be in favor of just rejecting a character set name as a client
encoding specification. I think inferring one is being generous.

-Kevin

On 18/04/11 15:57, Tom Lane wrote:

Bernd Helmle<mailings@oopsware.de> writes:

If i am reading it correct, it reads "UTF8" from the backend, while
expecting "UNICODE" only. Not sure what change has caused this,
though.

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over. For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UnIcOdE
(1 row)

versus HEAD:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
client_encoding
-----------------
UTF8
(1 row)

I wasn't aware that JDBC would fail on that. It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

regards, tom lane

Am I right in thinking that would be that change committed on the 7th
(http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ?
I've just run the JDBC test build on my machine and it fails dismally
with this very message repeated over and over again. What concerns me
most is that (assuming my dates are right) the JDBC driver has been
broken for 11 days and no one noticed. This would lead me to believe
that there is no JDBC build server. What would it take to set one up? If
someone can point me to a test machine I'd happily assist in setting one up.

As for the breakage itself I'm OK with a new driver version for a new
database version and from my experience people expect that. I recall a
number of people asking me if an 8.4 driver would be OK to use against 9
before the 9 version was stable.

Regards,

--
Mike Fowler
Registered Linux user: 379787

On Mon, Apr 18, 2011 at 11:14 AM, Mike Fowler <mike@mlfowler.com> wrote:

On 18/04/11 15:57, Tom Lane wrote:

Bernd Helmle<mailings@oopsware.de>  writes:

If i am reading it correct, it reads "UTF8" from the backend, while
expecting "UNICODE" only. Not sure what change has caused this,
though.

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over.  For instance, previous versions:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UnIcOdE
(1 row)

versus HEAD:

regression=# set client_encoding = 'UnIcOdE';
SET
regression=# show client_encoding ;
 client_encoding
-----------------
 UTF8
(1 row)

I wasn't aware that JDBC would fail on that.  It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

                       regards, tom lane

Am I right in thinking that would be that change committed on the 7th
(http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ?
I've just run the JDBC test build on my machine and it fails dismally with
this very message repeated over and over again. What concerns me most is
that (assuming my dates are right) the JDBC driver has been broken for 11
days and no one noticed. This would lead me to believe that there is no JDBC
build server. What would it take to set one up? If someone can point me to a
test machine I'd happily assist in setting one up.

As for the breakage itself I'm OK with a new driver version for a new
database version and from my experience people expect that. I recall a
number of people asking me if an 8.4 driver would be OK to use against 9
before the 9 version was stable.

Regards,

One would need a machine which supports java 1.4, 1.5, and 1.6 since
the driver builds all 3 versions. There's actually a 4th between 1.4
and 1.5 but I don't recall the specifics

Dave

Dave Cramer <pg@fastcrypt.com> writes:

On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wasn't aware that JDBC would fail on that. �It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

Older drivers will fail for sure. We can fix newer drivers, but if we
leave it we will see a slew of bug reports.

Yeah. I'm thinking what we should do here is revert the change, with a
note in the source about why, and also change the JDBC driver to send
and expect "UTF8" not "UNICODE" (which as Kevin says is more correct
anyway). Then in a few releases' time we can un-revert the server
change.

regards, tom lane

Mike Fowler <mike@mlfowler.com> writes:

On 18/04/11 15:57, Tom Lane wrote:

I am --- when I redid the GUC assign_hook logic a few weeks ago,
I changed the client_encoding code so that it shows the normalized
(official) name of the encoding, not whatever random string the client
sent over. For instance, previous versions:

Am I right in thinking that would be that change committed on the 7th
(http://archives.postgresql.org/pgsql-committers/2011-04/msg00039.php) ?

Yes, that one.

What concerns me
most is that (assuming my dates are right) the JDBC driver has been
broken for 11 days and no one noticed. This would lead me to believe
that there is no JDBC build server. What would it take to set one up?

+1 for doing something along that line.

regards, tom lane

On Mon, Apr 18, 2011 at 11:24 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Dave Cramer <pg@fastcrypt.com> writes:

On Mon, Apr 18, 2011 at 10:57 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wasn't aware that JDBC would fail on that.  It's pretty annoying that
it does, but maybe we should grin and bear it, ie revert the change to
canonicalize the GUC's value?

Older drivers will fail for sure. We can fix newer drivers, but if we
leave it we will see a slew of bug reports.

Yeah.  I'm thinking what we should do here is revert the change, with a
note in the source about why, and also change the JDBC driver to send
and expect "UTF8" not "UNICODE" (which as Kevin says is more correct
anyway).  Then in a few releases' time we can un-revert the server
change.

Well initially my concern was that people would have a challenge in
the case where they had to re-certify their application if we made
this change, however I realize they will have to do this anyway since
upgrading to 9.1 is what necessitates it.

So I'm less concerned with bug reports since people can just upgrade both

Dave Cramer

dave.cramer(at)credativ(dot)ca
http://www.credativ.ca

Dave Cramer <pg@fastcrypt.com> writes:

On Mon, Apr 18, 2011 at 11:24 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Yeah. I'm thinking what we should do here is revert the change, with a
note in the source about why, and also change the JDBC driver to send
and expect "UTF8" not "UNICODE" (which as Kevin says is more correct
anyway). Then in a few releases' time we can un-revert the server
change.

Well initially my concern was that people would have a challenge in
the case where they had to re-certify their application if we made
this change, however I realize they will have to do this anyway since
upgrading to 9.1 is what necessitates it.

I don't see any backwards compatibility risk, if that's what you mean.
Every backend release since 7.3 has treated client_encoding 'UTF8' and
'UNICODE' the same, and earlier releases didn't accept either one.

regards, tom lane

On 04/18/2011 11:25 AM, Tom Lane wrote:

What concerns me most is that (assuming my dates are right) the JDBC driver has been
broken for 11 days and no one noticed. This would lead me to believe
that there is no JDBC build server. What would it take to set one up?

+1 for doing something along that line.

All you'd need to do is write a step for a buildfarm animal to fetch the
JDBC driver and run some tests, and run it in a buildfarm client
somewhere. The server code is quite agnostic about the steps that are
reported on.

IOW in addition to a running buildfarm member you need to write a small
amount (< 100 lines, possibly much less) of perl code.

cheers

andrew

On 18/04/11 17:35, Andrew Dunstan wrote:

On 04/18/2011 11:25 AM, Tom Lane wrote:

What concerns me most is that (assuming my dates are right) the
JDBC driver has been
broken for 11 days and no one noticed. This would lead me to believe
that there is no JDBC build server. What would it take to set one up?

+1 for doing something along that line.

All you'd need to do is write a step for a buildfarm animal to fetch
the JDBC driver and run some tests, and run it in a buildfarm client
somewhere. The server code is quite agnostic about the steps that are
reported on.

IOW in addition to a running buildfarm member you need to write a
small amount (< 100 lines, possibly much less) of perl code.

cheers

andrew

I've found the entry on the Developer Wiki
(http://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto). What I'll
do is set-up three "farms" on my machine - one for 1.4, one for 1.5 and
one for 1.6. It's been a while since I've had an excuse to write some
Perl! I can't guarantee when I'll have it done as I'm away for a little
over a week from Wednesday and I'm not allowed internet access!

Regards,

--
Mike Fowler
Registered Linux user: 379787

On 18/04/11 17:12, Tom Lane wrote:

Dave Cramer<pg@fastcrypt.com> writes:

Well initially my concern was that people would have a challenge in
the case where they had to re-certify their application if we made
this change, however I realize they will have to do this anyway since
upgrading to 9.1 is what necessitates it.

I don't see any backwards compatibility risk, if that's what you mean.
Every backend release since 7.3 has treated client_encoding 'UTF8' and
'UNICODE' the same, and earlier releases didn't accept either one.

regards, tom lane

As there seems to be a consensus forming for fixing the JDBC driver,
I've taken the liberty do so at the risk of being shot down. The patch
is fairly straightforward, just changing UNICODE to UTF8 in a number of
files including the translation files. I've tested this against 9.1devel
(HEAD) and 8.4.7. For each database version I build and the tests
running JDKs 1.4.2_19, 1.5.0_22 and 1.6.0_2. All on 32-bit.

Regards,

--
Mike Fowler
Registered Linux user: 379787

On Mon, 18 Apr 2011, Mike Fowler wrote:

On 18/04/11 17:12, Tom Lane wrote:

Dave Cramer<pg@fastcrypt.com> writes:

Well initially my concern was that people would have a challenge in
the case where they had to re-certify their application if we made
this change, however I realize they will have to do this anyway since
upgrading to 9.1 is what necessitates it.

I don't see any backwards compatibility risk, if that's what you mean.
Every backend release since 7.3 has treated client_encoding 'UTF8' and
'UNICODE' the same, and earlier releases didn't accept either one.

regards, tom lane

As there seems to be a consensus forming for fixing the JDBC driver, I've
taken the liberty do so at the risk of being shot down. The patch is fairly
straightforward, just changing UNICODE to UTF8 in a number of files including
the translation files. I've tested this against 9.1devel (HEAD) and 8.4.7.
For each database version I build and the tests running JDKs 1.4.2_19,
1.5.0_22 and 1.6.0_2. All on 32-bit.

Thanks, applied, mostly. It's great to have a patch for a problem before
you even know it exists.

I didn't modify the .po files. I doubt this will change the adjacent
translation wording, but directly patching .po files is only something to
do in more dire circumstances (like needing to make a backpatch to an old
branch that won't get translators to look at it before the next release.)

I also discarded your changes to AbstractJdbc3Statement. Those Unicode
mentions are from the interface Javadoc, so I left them alone.

Kris Jurka