PG 9.0 and standard_conforming_strings

Bruce Momjian <bruce@momjian.us> wrote:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

If not now, when?

-Kevin

On Jan 29, 2010, at 11:51 AM, Bruce Momjian wrote:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

+1

David

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.
Changing that default will break, approximately speaking, every single
Postgres client app. Do you really think more than epsilon of them
are clean and ready for such a change?

regards, tom lane

I wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.

BTW, core already had that discussion, but maybe I should repeat it
to try to forestall any other "since this is going to be 9.0, let's
break backwards compatibility in a big way!" proposals. Now is not
the time to be making big changes; we are much too late in the devel
cycle to work through all the possible consequences. Because we
switched from it's-8.5 to it's-9.0 at such a late stage, we really
need to consider that that's only a marketing version number and
technical compatibility decisions should be made the same way as
for any other major release.

Perhaps at some point we will choose to do a major version bump where
we really do clean up a lot of bad backwards-compatibility things. That
needs to be done in a deliberate fashion with a lot of advance planning;
and things should get broken near the beginning of the devel cycle, not
the end.

[ still bearing scars from the 8.3 implicit-cast business, which we
didn't think would generate nearly the backlash it did... ]

regards, tom lane

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.
Changing that default will break, approximately speaking, every single
Postgres client app. Do you really think more than epsilon of them
are clean and ready for such a change?

Well, if they aren't ready now, then we might as well say we are never
going to change it and update the documentation and TODO list to reflect
that --- we have had standard_conforming_strings since 2005. We can't
keep pretending this will happen if we have no intention of doing it.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Jan 29, 2010 at 13:42, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.

[ still bearing scars from the 8.3 implicit-cast business, which we
didn't think would generate nearly the backlash it did... ]

Yeah that was my first reaction. But then again we also have a guc
they can change back. Sure you could create your own typecasts to
restore the old behavior in 8.3 (after trolling the mailing lists, or
finding some blog entry that got created X months after the
release...). Thats no where near as nice as a simple setting.

On Fri, Jan 29, 2010 at 3:28 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.
Changing that default will break, approximately speaking, every single
Postgres client app.  Do you really think more than epsilon of them
are clean and ready for such a change?

Well, I already had to fix a great many things in my apps to prevent
them from spewing warnings all over creation. If other people have
done likewise it might not be too bad; OTOH, there's probably not a
huge amount of downside in waiting.

...Robert

In response to Bruce Momjian <bruce@momjian.us>:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.
Changing that default will break, approximately speaking, every single
Postgres client app. Do you really think more than epsilon of them
are clean and ready for such a change?

Well, if they aren't ready now, then we might as well say we are never
going to change it and update the documentation and TODO list to reflect
that --- we have had standard_conforming_strings since 2005. We can't
keep pretending this will happen if we have no intention of doing it.

Announce it as a change for 9.1 NOW, and then it will be whoever's
fault if they aren't paying attention. Plenty of time to fix it
if it's announced now.

Also, as long as the config option is there, they can always flip it
back, which makes it MUCH lower overhead than the casting change was.
Overall, I don't think this change is nearly as severe as the cast
change in 8.3, and I don't feel it warrants the same eggshell walking.
When the decision is made to remove the standard_conforming_string
config option altogether ... that'll be a different story!

--
Bill Moran
http://www.potentialtech.com
http://people.collaborativefusion.com/~wmoran/

On Fri, 2010-01-29 at 15:45 -0500, Bruce Momjian wrote:

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.
Changing that default will break, approximately speaking, every single
Postgres client app. Do you really think more than epsilon of them
are clean and ready for such a change?

Well, if they aren't ready now, then we might as well say we are never
going to change it and update the documentation and TODO list to reflect
that --- we have had standard_conforming_strings since 2005. We can't
keep pretending this will happen if we have no intention of doing it.

I would argue that now is the perfect time for a number of reasons:

(1) 9.x regardless of the fact that it is just a number, reflects a
massive change as a whole.

(2) HS/SR are going to be scary things to use for at least 6 months to a
year. That is not to disparage the hard work, just that they are big
enough and invasive enough to make sure we get through a couple of dot
revs before we start seriously recommending them.

(3) As Bruce suggests, we are on year 6 now. I think we can take the
heat.

(4) The 8.3 issue wasn't nearly as bad as Tom is making it out to be.
Yes, there was a lot of WTF going on, but only by people that aren't
paying attention anyway and the work to fix it was pretty nominal.

(5) The time to change it is NOW, so that when we go into beta it
becomes a serious in your face, we have to fix this if we want to be
compatible with v9 of PostgreSQL.

And get this... because of HS and SR, everybody is going to want to be
compatible with v9.

Sincerely,

Joshua D. Drake

--
PostgreSQL.org Major Contributor
Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564
Consulting, Training, Support, Custom Development, Engineering
Respect is earned, not gained through arbitrary and repetitive use or Mr. or Sir.

Tom Lane wrote:

I wrote:

Bruce Momjian <bruce@momjian.us> writes:

With the release of Postgres 9.0, should we consider changing the
default for 'standard_conforming_strings'?

I'm inclined to think we're going to have enough problems without that.

BTW, core already had that discussion, but maybe I should repeat it
to try to forestall any other "since this is going to be 9.0, let's
break backwards compatibility in a big way!" proposals. Now is not
the time to be making big changes; we are much too late in the devel
cycle to work through all the possible consequences. Because we
switched from it's-8.5 to it's-9.0 at such a late stage, we really
need to consider that that's only a marketing version number and
technical compatibility decisions should be made the same way as
for any other major release.

Perhaps at some point we will choose to do a major version bump where
we really do clean up a lot of bad backwards-compatibility things. That
needs to be done in a deliberate fashion with a lot of advance planning;
and things should get broken near the beginning of the devel cycle, not
the end.

[ still bearing scars from the 8.3 implicit-cast business, which we
didn't think would generate nearly the backlash it did... ]

I did ask this same question for the 8.5/9.0 release in April of 2009 so
don't say I am only asking about this at the end of development cycles:

http://archives.postgresql.org/pgsql-hackers/2009-04/msg00512.php

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Joshua D. Drake wrote:

(4) The 8.3 issue wasn't nearly as bad as Tom is making it out to be.
Yes, there was a lot of WTF going on, but only by people that aren't
paying attention anyway and the work to fix it was pretty nominal.

The big mistake we made in 8.3 is not having those compatibility
functions that Peter created ready _at_ _release_ _time_. I believe
that was pure sloppiness on our part.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alex Hunsaker <badalex@gmail.com> writes:

On Fri, Jan 29, 2010 at 13:42, Tom Lane <tgl@sss.pgh.pa.us> wrote:

[ still bearing scars from the 8.3 implicit-cast business, which we
didn't think would generate nearly the backlash it did... ]

Yeah that was my first reaction. But then again we also have a guc
they can change back.

"There's a GUC for it" is NOT a helpful answer; if there's one thing
that we've learned the hard way over the past years, it's that GUCs
don't solve compatibility problems. Applications don't know to set
them, and having the wrong setting can easily become a security hole
(particularly for this one).

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

regards, tom lane

Tom Lane wrote:

Alex Hunsaker <badalex@gmail.com> writes:

On Fri, Jan 29, 2010 at 13:42, Tom Lane <tgl@sss.pgh.pa.us> wrote:

[ still bearing scars from the 8.3 implicit-cast business, which we
didn't think would generate nearly the backlash it did... ]

Yeah that was my first reaction. But then again we also have a guc
they can change back.

"There's a GUC for it" is NOT a helpful answer; if there's one thing
that we've learned the hard way over the past years, it's that GUCs
don't solve compatibility problems. Applications don't know to set
them, and having the wrong setting can easily become a security hole
(particularly for this one).

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

Well, since I asked in April of 2009, at the beginning of the cycle, 6
years after the introduction of the variable, and we still are not doing
it, then let's stop pretending we will ever do it.

The way the docs stand now we hold it over people's heads and issue
warnings that are meaningless if we are never going to change it.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

On Fri, Jan 29, 2010 at 14:03, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Alex Hunsaker <badalex@gmail.com> writes:

On Fri, Jan 29, 2010 at 13:42, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

After skimming the thread Bruce linked:
http://archives.postgresql.org/pgsql-hackers/2009-04/msg00512.php

It certainly seems "insufficiently-thought-out". :(

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

I don't see how announcing this earlier in the dev cycle would help, at
all. The people who read -hackers have been using
standards-conforming-strings for years. Further, if we announce it now,
people have 4-5 months to get ready for it, assuming they were updating
to 9.0.0 anyway, which I doubt anyone is.

For this release, I'm already planning to have big "backwards
compatibility" section and web page with *lots* of warnings and
explanations, and because of the media around the release for once it
will be read.

I'd argue that Bruce is right; if we're not going to do it now, we might
as well stop pretending we ever are.

--Josh Berkus

Alex Hunsaker wrote:

On Fri, Jan 29, 2010 at 14:03, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Alex Hunsaker <badalex@gmail.com> writes:

On Fri, Jan 29, 2010 at 13:42, Tom Lane <tgl@sss.pgh.pa.us> wrote:

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

After skimming the thread Bruce linked:
http://archives.postgresql.org/pgsql-hackers/2009-04/msg00512.php

It certainly seems "insufficiently-thought-out". :(

Is this still true? When we changed plpgsql so it shared the scanner
with the backend scanner, does this issue no longer apply, i.e.
consider honoring standard_conforming_strings in PL/pgSQL function
bodies?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alex Hunsaker <badalex@gmail.com> wrote:

After skimming the thread Bruce linked:
http://archives.postgresql.org/pgsql-hackers/2009-04/msg00512.php

It certainly seems "insufficiently-thought-out". :(

Just as a clarification, while the GUC was *added* in 8.1, it was
read-only with a value of 'off'. I submitted a patch and started
using it under 8.1 in February of 2006 (because we had an urgent
need), and it officially became *settable* in 8.2.

I don't have strong feelings about changing the default. Obviously,
this bites people primarily when converting to PostgreSQL -- that's
when it bit me and that's where people normally are when they post
to the lists about related issues.

It's not clear to me that the issues related to functions have been
thought out sufficiently; my personal feeling is that a function
should run with the setting under which it was created (as the
semantics of the literal seem as though they should be "frozen" at
that point), but that was shot down. And then there's the issue
about EXECUTE. If we don't have consensus on a solution to those
issues, maybe we should wait. Those who need it who are already
using PostgreSQL already have it figured out -- it's just a bump on
the road to converting for those used to standard literals.

-Kevin

Josh Berkus <josh@agliodbs.com> writes:

I stand by the position that it's way too late in the cycle for
insufficiently-thought-out proposals for major behavioral changes.

I don't see how announcing this earlier in the dev cycle would help, at
all.

We would have more than no-time-at-all to test it and fix any breakage.
Just to start close to home, do you really trust either psql or pg_dump
to be completely free of standard_conforming_strings issues? How about
JDBC or ODBC? Python drivers? PLs?

The really short and sweet answer is that if you have any ambition at
all to ship 9.0 this year, it is too late to add new work items. This
is a work item, and not a small one.

regards, tom lane

Bruce Momjian <bruce@momjian.us> writes:

Well, since I asked in April of 2009, at the beginning of the cycle, 6
years after the introduction of the variable, and we still are not doing
it, then let's stop pretending we will ever do it.

We have made forward progress since that thread (we fixed the plpgsql
parsing issues, partially in 8.4 and completely for 9.0). We can
continue to make forward progress in the future. But *right now is not
the time*. We have more than enough on our plates for 9.0 already, and
just about nobody believes we are going to meet the schedule now.

regards, tom lane