Avoiding upgrade backlash

Josh Berkus <josh@agliodbs.com> writes:

While we are right to choke down abuse of implicit casting, it might be a
good idea to give users stronger warnings and instructions.

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade (this is
always true but now doing it wrong will break a lot more users).

Uh, what makes you think that will make any difference?

3) If Robert gets his type-cast backport package together, the location of
that.

The very *last* thing I want to see is encouragement of people trying to
undo that change.

regards, tom lane

On Mon, Nov 12, 2007 at 02:42:30PM -0800, Josh Berkus wrote:

While we are right to choke down abuse of implicit casting, it might be a
good idea to give users stronger warnings and instructions.

I think this is true. I think it maybe ought to be right at the top of the
notes, and maybe ought to be linked off the announcement of the sort

Users upgrading from previous releases want to read this <a etc>
compatibility note</a>

I think maybe how Debian handled the notes about kernel changes in the last
stable release would be a good model. The point there is, _test well_.
We're doing what we warned you about.

I'm thinking that we need to warn everyone about:

[&c]

This list seems to miss the main warning needed, which is this in 300 point
bold:

IMPLICIT TYPECASTS HAVE BEEN MOSTLY REMOVED. TEST YOUR APPLICATION!

A
--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

"Andrew Sullivan" <ajs@crankycanuck.ca> writes:

IMPLICIT TYPECASTS HAVE BEEN MOSTLY REMOVED. TEST YOUR APPLICATION!

I think you have to say that without using the technical gobbledygook
"implicit casts".

Something like:

Non-strings are no longer converted automatically to strings when used in
places where strings are expected. You must now explicitly cast non-strings to
strings if you want to apply string operations on them.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's RemoteDBA services!

Gregory Stark wrote:

"Andrew Sullivan" <ajs@crankycanuck.ca> writes:

IMPLICIT TYPECASTS HAVE BEEN MOSTLY REMOVED. TEST YOUR APPLICATION!

I think you have to say that without using the technical gobbledygook
"implicit casts".

Something like:

Non-strings are no longer converted automatically to strings when used in
places where strings are expected. You must now explicitly cast non-strings to
strings if you want to apply string operations on them.

I think you need to show actual examples of problem queries.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Am Montag, 12. November 2007 schrieb Josh Berkus:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade (this is
always true but now doing it wrong will break a lot more users).

What difference would pg_dump make?

2) They need to check for bugs

What bugs?

3) If Robert gets his type-cast backport package together, the location of
that.

Well, if you want to undo the changes, you don't need a backport package; you
can just change the cast's definition.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:

Am Montag, 12. November 2007 schrieb Josh Berkus:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade (this is
always true but now doing it wrong will break a lot more users).

What difference would pg_dump make?

well the current problem is that either way pg_dump can generate dumps
that are not restorable without modification on 8.3 (like say one that
contains a VIEW that does a INTEGER = TEXT comparision which 8.3 will
refuse to create) even if dumped with 8.3.

2) They need to check for bugs

What bugs?

queries that depend on implict casts "just working" - most of those are
actually bug or maybe sloppy coding on the application side but it is
imho by FAR the most incompatible behaviour change we have done in the
last few releases.
I guess that the largest amount of these are TEXT/INTEGER casts and the
kind of app that will probably hit most are the ones that are
misbehaving anyway (think EAV style stuff) - but it is a incompatible
change nevertheless.

3) If Robert gets his type-cast backport package together, the location of
that.

Well, if you want to undo the changes, you don't need a backport package; you
can just change the cast's definition.

well the point here is that we ought to be more open about the impact of
the change and i think that the release notes need to mention that (and
maybe also an example on how to change/re-add the casts)

Stefan

On Mon, Nov 12, 2007 at 11:12:52PM -0500, Bruce Momjian wrote:

I think you need to show actual examples of problem queries.

Good idea.

--
Andrew Sullivan
Old sigs will return after re-constitution of blue smoke

On Monday 12. November 2007, Josh Berkus wrote:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade

Isn't there a chicken-and-egg problem here? On my Gentoo system, I have
always been forced to uninstall the old version before upgrading to a
new version. In practice, that means that I have to do a full dump of
the databases with the current pg_dump, delete or rename the data
directory, and then restore after the upgrade.

I'd like to see a detailed howto on this ...
--
Leif Biberg Kristensen | Registered Linux User #338009
http://solumslekt.org/ | Cruising with Gentoo/KDE
My Jazz Jukebox: http://www.last.fm/user/leifbk/

Leif B. Kristensen wrote:

On Monday 12. November 2007, Josh Berkus wrote:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade

Isn't there a chicken-and-egg problem here? On my Gentoo system, I have
always been forced to uninstall the old version before upgrading to a
new version.

That's a Gentoo shortcoming, nothing more.

--
Alvaro Herrera http://www.advogato.org/person/alvherre
"Endurecerse, pero jam�s perder la ternura" (E. Guevara)

Peter,

What difference would pg_dump make?

For many user's databases ... many more than any previous version after
7.3 ... using the old version's pg_dump *will not load* into 8.3. Unless you
restore the casts.

Well, if you want to undo the changes, you don't need a backport package;
you can just change the cast's definition.

Right. We've talked (on IRC) about putting together a package which restores
the removed casts.

Given that Gentoo, Red Hat and SuSE don't support having multiple PostgreSQL
versions on the system in their packaging, I'm thinking the "restore casts"
package is essential rather than being a nice idea.

Should I be discussing this on a different list?

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

Am Dienstag, 13. November 2007 schrieb Josh Berkus:

What difference would pg_dump make?

For many user's databases ... many more than any previous version after
7.3 ... using the old version's pg_dump *will not load* into 8.3. �Unless
you restore the casts.

An example case would be useful here as well.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

--On Dienstag, November 13, 2007 11:05:18 +0100 Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:

What difference would pg_dump make?

well the current problem is that either way pg_dump can generate dumps
that are not restorable without modification on 8.3 (like say one that
contains a VIEW that does a INTEGER = TEXT comparision which 8.3 will
refuse to create) even if dumped with 8.3.

AFAIK pg_dump and its related dump routines takes care to dump this in a
proper way with explicit casts attached. Do you have an example showing a
possible misbehavior?

--
Thanks

Bernd

Josh Berkus <josh@agliodbs.com> writes:

Peter,

What difference would pg_dump make?

For many user's databases ... many more than any previous version after
7.3 ... using the old version's pg_dump *will not load* into 8.3. Unless you
restore the casts.

Please provide a concrete example.

Should I be discussing this on a different list?

If you are looking for code changes, neither docs nor advocacy are
suitable forums.

regards, tom lane

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 13 Nov 2007 11:39:26 -0500
Tom Lane <tgl@sss.pgh.pa.us> wrote:

Josh Berkus <josh@agliodbs.com> writes:

Peter,

What difference would pg_dump make?

For many user's databases ... many more than any previous version
after 7.3 ... using the old version's pg_dump *will not load* into
8.3. Unless you restore the casts.

Please provide a concrete example.

Should I be discussing this on a different list?

If you are looking for code changes, neither docs nor advocacy are
suitable forums.

Josh wasn't looking for code changes. He was insuring that we were loud
about the rather significant problems that upgrading to 8.3, "may"
cause.

Joshua D. Drake

regards, tom lane

---------------------------(end of
broadcast)--------------------------- TIP 4: Have you searched our
list archives?

http://archives.postgresql.org

- --

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHOdWAATb/zqfZUUQRAjG4AJ9WZgxRABLBu/E6gid1t1Jg1pDxcQCfanaB
MNH1nlHU6lEO+z+IJ6mxLvE=
=2Tga
-----END PGP SIGNATURE-----

Peter Eisentraut <peter_e@gmx.net> writes:

Am Montag, 12. November 2007 schrieb Josh Berkus:

3) If Robert gets his type-cast backport package together, the location of
that.

Well, if you want to undo the changes, you don't need a backport
package; you can just change the cast's definition.

It's actually not going to be that easy, because most of those casts
aren't even in pg_cast anymore: they have been subsumed into the
CoerceViaIO mechanism. You'd need to resurrect the individual cast
functions before you could put entries back, too.

Another little problem is that you're likely to break as much stuff as
you fix. An example in CVS HEAD:

regression=# select 42 || 'foo';
?column?
----------
42foo
(1 row)

regression=# select 42 like 'foo';
ERROR: operator does not exist: integer ~~ unknown
LINE 1: select 42 like 'foo';
^
HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts.

OK, let's "fix" that by making int->text implicit again:

regression=# create function inttotext(int) returns text as $$
regression$# begin return $1; end$$ language plpgsql strict immutable;
CREATE FUNCTION
regression=# create cast (int as text) with function inttotext(int)
regression-# as implicit;
CREATE CAST

Now LIKE works:

regression=# select 42 like 'foo';
?column?
----------
f
(1 row)

but || not so much:

regression=# select 42 || 'foo';
ERROR: operator is not unique: integer || unknown
LINE 1: select 42 || 'foo';
^
HINT: Could not choose a best candidate operator. You might need to add explicit type casts.

regards, tom lane

On Tuesday 13. November 2007, Alvaro Herrera wrote:

Leif B. Kristensen wrote:

On Monday 12. November 2007, Josh Berkus wrote:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade

Isn't there a chicken-and-egg problem here? On my Gentoo system, I
have always been forced to uninstall the old version before
upgrading to a new version.

That's a Gentoo shortcoming, nothing more.

I've started a thread about it in the Portage & Programming section of
forums.gentoo.org:

http://forums.gentoo.org/viewtopic-t-615494.html

I hope that someone more knowledgeable than me will be able to
contribute in bringing this issue to the attention of the Gentoo
developers, and hopefully work out a solution. Sure, there are other
Gentoo users than me around here?
--
Leif Biberg Kristensen | Registered Linux User #338009
http://solumslekt.org/ | Cruising with Gentoo/KDE
My Jazz Jukebox: http://www.last.fm/user/leifbk/

On Wednesday 14. November 2007, Leif B. Kristensen wrote:

On Tuesday 13. November 2007, Alvaro Herrera wrote:

Leif B. Kristensen wrote:

On Monday 12. November 2007, Josh Berkus wrote:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade

Isn't there a chicken-and-egg problem here? On my Gentoo system, I
have always been forced to uninstall the old version before
upgrading to a new version.

That's a Gentoo shortcoming, nothing more.

I've started a thread about it in the Portage & Programming section of
forums.gentoo.org:

http://forums.gentoo.org/viewtopic-t-615494.html

I hope that someone more knowledgeable than me will be able to
contribute in bringing this issue to the attention of the Gentoo
developers, and hopefully work out a solution. Sure, there are other
Gentoo users than me around here?

FYI, I've filed a bug report on bugs.gentoo.org:

"This is not quite a bug, but rather an approaching challenge. See this
thread on forums.gentoo.org
[http://forums.gentoo.org/viewtopic-t-615494.html], in which I was
asked to file a bug report. Concerning the upcoming PostgreSQL upgrade
from 8.2 to 8.3, it's imperative that Gentoo implements an adequate
transition procedure. Blocking the old version will create
a 'chicken-and-egg' situation where you are left incapable of upgrading
your existing databases. According to the developers, it's important
that a dump of the old database should be done with the new 8.3 version
of pg_dump. See this thread
[http://archives.postgresql.org/pgsql-advocacy/2007-11/msg00051.php] on
pgsql-advocacy for some details."

The bug id is [https://bugs.gentoo.org/show_bug.cgi?id=199142].
--
Leif Biberg Kristensen | Registered Linux User #338009
http://solumslekt.org/ | Cruising with Gentoo/KDE
My Jazz Jukebox: http://www.last.fm/user/leifbk/

Am Mittwoch, 14. November 2007 schrieb Leif B. Kristensen:

from 8.2 to 8.3, it's imperative that Gentoo implements an adequate

transition procedure. Blocking the old version will create
a 'chicken-and-egg' situation where you are left incapable of upgrading
your existing databases. According to the developers, it's important
that a dump of the old database should be done with the new 8.3 version
of pg_dump.

Note that this is actually not true on both accounts. Firstly, no developer
has ever claimed that, and second it's factually false.

It has always been "better" to use the newer pg_dump, but never "important" or
imperative.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Wednesday 14. November 2007, Peter Eisentraut wrote:

Am Mittwoch, 14. November 2007 schrieb Leif B. Kristensen:

from 8.2 to 8.3, it's imperative that Gentoo implements an adequate

transition procedure. Blocking the old version will create
a 'chicken-and-egg' situation where you are left incapable of
upgrading your existing databases. According to the developers, it's
important that a dump of the old database should be done with the
new 8.3 version of pg_dump.

Note that this is actually not true on both accounts. Firstly, no
developer has ever claimed that, and second it's factually false.

It has always been "better" to use the newer pg_dump, but never
"important" or imperative.

Here is what Josh said when he started this thread:

On Monday 12. November 2007, Josh Berkus wrote:

I'm thinking that we need to warn everyone about:
1) They need to use 8.3's pg_dump, not the old version, to upgrade

Repeat: "They need to use 8.3's pg_dump, not the old version, to
upgrade"

Do you mean to say that Josh is lying?
--
Leif Biberg Kristensen | Registered Linux User #338009
http://solumslekt.org/ | Cruising with Gentoo/KDE
My Jazz Jukebox: http://www.last.fm/user/leifbk/