Thoughts on maintaining 7.3

On Tue, 30 Sep 2003, Joshua D. Drake wrote:

Hello,

With the recent stint of pg_upgrade statements and the impending
release of 7.4 what do people think about having a dedicated maintenance
team for 7.3? 7.3 is a pretty solid release and I think people will be
hard pressed to upgrade to 7.4. Of course a lot of people will, but I
have customer that are just now upgrading to 7.3 because of legacy
application and migratory issues.

Anyway I was considering a similar situation to how Linux works where
their is a maintainer for each release... Heck even Linux 2.0 still
released until recently.

Of course the theory being that we backport "some" features and fix
any bugs that we find?

What are people's thoughts on this?

The key issue here is that those creating the patches need to spend the
time to create appropriate ones for v7.3, and not many seem willing ...
Tom generally does alot of work on back-patching where appropriate, but
those patches are generally either very critical, or benign to changes
since v7.3 ...

The main detractor from us doing this up to this point has been, I
believe, testing to make sure any back patches don't break *any* of the
various OS ports, testing that generally only gets done while in a Beta
freeze ...

Not saying that if someone submit'd patches to v7.3, they wouldn't get
applied ... only that, to date, the work/effort has been greater then the
overall benefit, and nobody has step'd up to the plate to do it ...

On Wed, 2003-10-01 at 08:36, Marc G. Fournier wrote:

On Tue, 30 Sep 2003, Joshua D. Drake wrote:

Hello,

With the recent stint of pg_upgrade statements and the impending
release of 7.4 what do people think about having a dedicated maintenance
team for 7.3? 7.3 is a pretty solid release and I think people will be
hard pressed to upgrade to 7.4. Of course a lot of people will, but I
have customer that are just now upgrading to 7.3 because of legacy
application and migratory issues.

Anyway I was considering a similar situation to how Linux works where
their is a maintainer for each release... Heck even Linux 2.0 still
released until recently.

Of course the theory being that we backport "some" features and fix
any bugs that we find?

What are people's thoughts on this?

The key issue here is that those creating the patches need to spend the
time to create appropriate ones for v7.3, and not many seem willing ...
Tom generally does alot of work on back-patching where appropriate, but
those patches are generally either very critical, or benign to changes
since v7.3 ...

The main detractor from us doing this up to this point has been, I
believe, testing to make sure any back patches don't break *any* of the
various OS ports, testing that generally only gets done while in a Beta
freeze ...

Not saying that if someone submit'd patches to v7.3, they wouldn't get
applied ... only that, to date, the work/effort has been greater then the
overall benefit, and nobody has step'd up to the plate to do it ...

Maybe I've mis-read Joshua's intentions, but I got the impression that
this 7.3 maintainer would follow the patches list and backport patches
whenever possible. This way folks coding for 7.4/7.5 can stay focused on
that, but folks who can't upgrade to 7.4 for whatever reason can still
get some features / improvements.

Several linux distros already do this for many packages, and personally
I've always been surprised that, given postgresql's major release
upgrade issues, that no commercial company has stepped in to offer this
in the past. I think what Joshua is wondering is how much cooperation
would he get from the community if he was willing to donate these
efforts back into project.

While your concerns about testing are valid, there are already issues
with that for minor releases, as evidenced by our need to do the quick
7.3.4 after trouble in 7.3.3. Not to mention how little testing is
happening to the code that's been back patched into 7.3 since 7.3.4...
Hmm... maybe thats actually an argument against having more changes get
put in, OTOH if Joshua can address the testing issues maybe there would
be an overall improvement.

I personally think it's a good idea for *someone* to do this, but I'll
leave it to core to decide if they want to put the projects stamp of
approval on it for any official community release.

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Tue, 30 Sep 2003, Joshua D. Drake wrote:

Hello,

With the recent stint of pg_upgrade statements and the impending
release of 7.4 what
do people think about having a dedicated maintenance team for 7.3? 7.3
is a pretty
solid release and I think people will be hard pressed to upgrade to 7.4.
Of course
a lot of people will, but I have customer that are just now upgrading to
7.3 because
of legacy application and migratory issues.

Anyway I was considering a similar situation to how Linux works where
their is a
maintainer for each release... Heck even Linux 2.0 still released until
recently.

Of course the theory being that we backport "some" features and fix
any bugs that
we find?

What are people's thoughts on this?

It seems to me the upgrade from 7.2 to 7.4 is easier than an upgrade to
7.3, since at least 7.4's pg_dumpall can connect to a 7.2 database and
suck in everything, whereas in 7.3 I had to dump with 7.2's dumpall and
then tweak the file by hand a fair bit to get it to go into 7.3.

With 7.4 I'm finding upgrading to be easier. I'll likely upgrade out
production servers to 7.4.0 when it comes out and wind up skipping 7.3
altogether.

On Wed, 1 Oct 2003, Robert Treat wrote:

Maybe I've mis-read Joshua's intentions, but I got the impression that
this 7.3 maintainer would follow the patches list and backport patches
whenever possible. This way folks coding for 7.4/7.5 can stay focused on
that, but folks who can't upgrade to 7.4 for whatever reason can still
get some features / improvements.

The problem, I think (and please note that I'm not against it, just
playing major devil's advocate here) is that there have always been some
major fundamental coding changes between releases that there are very few
patches that are "back-patchable" without having to do some heavy
re-writes ...

Several linux distros already do this for many packages, and personally
I've always been surprised that, given postgresql's major release
upgrade issues, that no commercial company has stepped in to offer this
in the past. I think what Joshua is wondering is how much cooperation
would he get from the community if he was willing to donate these
efforts back into project.

Using Linux/FreeBS/Insert OS Here as an example is like comparing apples
to oranges ... take FreeBSD as an example, since I know it ... 5.x has had
some *major* re-writes to the kernel done to it, getting rid of 'the Giant
Lock' that SMP in 4.x uses ... those changes are not back-patchable, since
then you'd have 5.x ... there are alot of changes to the 5.x kernel that
rely on those changes, and are therefore not *easily* back-patchable ...

Now, userland software is a totally different case, since they are rarely
"tied" to the kernel itself ...

Think of PostgreSQL as the kernel, not as the distro ... how many changes
from one kernel release ae easily patched into an older one, without
having to take alot of other baggage back with it ... ?

I personally think it's a good idea for *someone* to do this, but I'll
leave it to core to decide if they want to put the projects stamp of
approval on it for any official community release.

I don't believe anyone would work against this, nor could I imagine that
anyone would think it was "a bad idea", I'm just curious as to how
possible it is to do ...

"Marc G. Fournier" <scrappy@postgresql.org> writes:

On Tue, 30 Sep 2003, Joshua D. Drake wrote:

Of course the theory being that we backport "some" features and fix
any bugs that we find?

Not saying that if someone submit'd patches to v7.3, they wouldn't get
applied ... only that, to date, the work/effort has been greater then the
overall benefit, and nobody has step'd up to the plate to do it ...

The idea of backporting features scares me; I really doubt that you can
get enough beta-testing on a back branch to be confident that you
haven't broken anything with a feature addition. In any case you'd be
quite limited in what you could do without forcing an initdb.

Another issue is that people expect dot-releases to be absolutely rock
solid. If you start introducing new features then you considerably
increase the risk of introducing new bugs. (I'm still embarrassed about
7.3.3's failure-to-start bug...)

Our past practice has been to back-port only bug fixes, and only
critical or low-risk ones at that. I think this could be done in a more
thorough fashion, and it could be continued longer than we've done in
the past, but you shouldn't set the scope of the maintenance effort any
wider than that.

regards, tom lane

On Wed, 2003-10-01 at 09:14, Robert Treat wrote:

Maybe I've mis-read Joshua's intentions, but I got the impression that
this 7.3 maintainer would follow the patches list and backport patches
whenever possible. This way folks coding for 7.4/7.5 can stay focused on
that, but folks who can't upgrade to 7.4 for whatever reason can still
get some features / improvements.

I don't think there's a need for a formalized "7.3 maintainer" -- if
individuals would like to see particular fixes backported to 7.3, they
can read pgsql-patches and post backported patches themselves. If
someone wants to go ahead and do that, I wouldn't complain. (Similarly,
if there is enough demand for a commercial company to do something
similar for their customers, that might also be a good idea).

However, I think it's a bad idea to backport any features into older
releases. The reason 7.3.x is really stable is precisely that it has had
a lot of testing and bugfixing work done, but no new features.
Furthermore, adding more features to 7.3.x reduces the incentive to
upgrade to 7.4, worsening the support problem: the more people using old
releases, the more demand there will be for backported features, leading
to more people using 7.3, leading to more demand for ...

(FWIW, I think that any energy we might spend on a 7.3 maintainer would
be better directed at improving the upgrade story...)

-Neil

On Wed, 2003-10-01 at 10:49, Neil Conway wrote:

On Wed, 2003-10-01 at 09:14, Robert Treat wrote:

Maybe I've mis-read Joshua's intentions, but I got the impression that
this 7.3 maintainer would follow the patches list and backport patches
whenever possible. This way folks coding for 7.4/7.5 can stay focused on
that, but folks who can't upgrade to 7.4 for whatever reason can still
get some features / improvements.

I don't think there's a need for a formalized "7.3 maintainer" -- if
individuals would like to see particular fixes backported to 7.3, they
can read pgsql-patches and post backported patches themselves. If
someone wants to go ahead and do that, I wouldn't complain. (Similarly,
if there is enough demand for a commercial company to do something
similar for their customers, that might also be a good idea).

ok

However, I think it's a bad idea to backport any features into older
releases. The reason 7.3.x is really stable is precisely that it has had
a lot of testing and bugfixing work done, but no new features.

eh.. i could see some things, like tsearch2 or pg_autovacuum, which
afaik are almost if not completely compatible with 7.3, which will not
get back ported. Also fixes in some of the extra tools like psql could
be very doable, I know I had a custom psql for 7.2 that back patched the
\timing option and some of the pager fixes. now, weather that could be
done with stuff closer to core, i don't know...

btw personally i'm fine with these things not being packpatched, though
if someone came out with a 7.3 pg_autovacuum rpm, or a 7.3 psql rpm, i'm
sure a lot of people would use it.

Furthermore, adding more features to 7.3.x reduces the incentive to
upgrade to 7.4, worsening the support problem: the more people using old
releases, the more demand there will be for backported features, leading
to more people using 7.3, leading to more demand for ...

(FWIW, I think that any energy we might spend on a 7.3 maintainer would
be better directed at improving the upgrade story...)

<homer>mmm. in place upgrade</homer>

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Tue, Sep 30, 2003 at 09:37:26AM -0700, Joshua D. Drake wrote:

Of course the theory being that we backport "some" features and fix
any bugs that
we find?

I would argue _very strongly_ against backporting features.

The backporting of features into the Linux kernel is an extremely
good analogy in this case. Someone gets the clever idea that this or
that feature from 2.1/2.3/2.5 is desperately needed in 2.0/2.2/2.4
and merrily goes about adding all sorts of new cruft to the so-called
stable release. As a result, we have plenty of examples of massive
filesystem corruption, modules that used to work and just plain don't
any more, sudden surprise hardware incompatibilites, &c. All too
frequently releases in the "stable" series are one right atop the
other. What's worse, all these additional features are bound up with
the important remote-root-type patches that make it into later
releases of the kernel. As a result, it's a lot of work to compile a
known-safe and known-clean kernel for use on one's own machines.

Patching an older release to fix critical, data-mangling bugs is one
thing. But if people want the latest nifty feature backported to an
old release, let 'em pay the developer to do it in their private
source tree, and not force on the rest of us the job of sorting out
what crucial patches we need to apply to our old, pristine source of
PostgreSQL 7.3.4. If you're really going to trust your database
software, you do not allow new features to be added after having
carefully teated all your applications against the system.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

eh.. i could see some things, like tsearch2 or pg_autovacuum, which
afaik are almost if not completely compatible with 7.3, which will not
get back ported. Also fixes in some of the extra tools like psql could
be very doable, I know I had a custom psql for 7.2 that back patched the
\timing option and some of the pager fixes. now, weather that could be
done with stuff closer to core, i don't know...

Sure but businesses don't like to upgrade unless they have too. If we
really want to attract more business to using PostgreSQL then they need
to feel like they don't have to upgrade every 12 months. Upgrading is
expensive and it rarely goes as smoothly as a dump/restore.

Furthermore, adding more features to 7.3.x reduces the incentive to
upgrade to 7.4, worsening the support problem: the more people using old
releases, the more demand there will be for backported features, leading
to more people using 7.3, leading to more demand for ...

I am considering a time limited type thing. Not open ended. Something like
18 or 24 months (max) from release of the new version. You can't expect
business to consider that timeframe during the development of the new
release. They want to see the new release in action for a period of time.
They also want time to play with the new release without sacrificing
support for the previous release.

<homer>mmm. in place upgrade</homer>

In reality in place upgrade will never work. Sure we can build a script
that will deal with PostgreSQL itself, but not user defined data types,
operators, functions etc... Those are all things that need stable time to
migrate and test.

Sincerely,

Joshua Drake

Robert Treat

--
Co-Founder
Command Prompt, Inc.
The wheel's spinning but the hamster's dead

I would argue _very strongly_ against backporting features.

For massive features sure but an example of a feature that works
very well and easily with 7.3 is the preloading of libs.

Sincerely,

Joshua Drake

--
Co-Founder
Command Prompt, Inc.
The wheel's spinning but the hamster's dead

On Wed, 2003-10-01 at 09:41, Marc G. Fournier wrote:

On Wed, 1 Oct 2003, Robert Treat wrote:

Several linux distros already do this for many packages, and personally
I've always been surprised that, given postgresql's major release
upgrade issues, that no commercial company has stepped in to offer this
in the past. I think what Joshua is wondering is how much cooperation
would he get from the community if he was willing to donate these
efforts back into project.

Using Linux/FreeBS/Insert OS Here as an example is like comparing apples
to oranges ... take FreeBSD as an example, since I know it ... 5.x has had
some *major* re-writes to the kernel done to it, getting rid of 'the Giant
Lock' that SMP in 4.x uses ... those changes are not back-patchable, since
then you'd have 5.x ... there are alot of changes to the 5.x kernel that
rely on those changes, and are therefore not *easily* back-patchable ...

Now, userland software is a totally different case, since they are rarely
"tied" to the kernel itself ...

you missed my point. some distro's (red hat, suse, mandrake, etc..)
backpatch into their distributed packages separate of the packages
original source tree. this is great for folks who may want/need a new
change, but can't upgrade to latest source for some reason.

Think of PostgreSQL as the kernel, not as the distro ... how many changes
from one kernel release ae easily patched into an older one, without
having to take alot of other baggage back with it ... ?

I wasn't thinking of PostgreSQL as a distro, but actually I think that
view is somewhat valid, since there are enough add ons to core that one
could modify without having to make huge changes.

As Tom pointed out, with the restriction of not being able to initdb,
you're probably pretty limited on what you can push back, but I think
there's still enough there that folks might want to look at it. (The
recent bugs in pltcl handling dropped columns come to mind, though maybe
Tom backpatched those? Cant recall)

Robert Treat
--
Build A Brighter Lamp :: Linux Apache {middleware} PostgreSQL

On Wed, Oct 01, 2003 at 08:49:51AM -0700, Joshua D. Drake wrote:

I would argue _very strongly_ against backporting features.

For massive features sure but an example of a feature that works
very well and easily with 7.3 is the preloading of libs.

Then let people patch the stable releases themselves, or pay
companies to produce such mini-branches (and thereby pay the cost of
the necessary testing, &c.).

How does one know in advance which set of "working well and easily"
features can be back ported and be sure not to break on some release
of IRIX, Solaris, AIX, or SCO? Those are not platforms that get the
kind of kicking that Linux and FreeBSD do, but people are still
relying on the dot releases not to break anything on those platforms.
I think that Postgres has a tradition that, when a release is stable,
it's _stable, man_ -- a tradition that other software (commercial or not)
should emulate. I'd hate to see that go overboard in an attempt to
add features to the main releases.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

On Wed, 2003-10-01 at 11:48, Joshua D. Drake wrote:

Sure but businesses don't like to upgrade unless they have too.

Granted, but maintaining old releases doesn't come at zero cost. It may
benefit some users, but the relevant question is whether that benefit is
worth the cost. The time someone spends backpatching changes into old
releases (and thoroughly testing those changes, and fixing the
regressions those changes cause) is presumably time that would otherwise
be spent improving the latest release of PostgreSQL.

So when the bugfix is important, has been well-tested, and is unlikely
to cause regressions, backpatching the change to previous stable
releases is a good idea. When this isn't the case (and even more so if
it's a feature and not a bugfix), I don't think it justifies the cost
(and the risk of destabilization) for most users.

In summary, I think the status quo is basically okay. Perhaps we should
backpatch a few more things, but we're basically in the right ballpark.

In reality in place upgrade will never work.

Perhaps not, but the upgrade story can certainly be made more palatable.
I think that's the actual problem here -- rather than skating around it
by making it less necessary to do the upgrade in the first place, I
think our time is better spent making upgrades as painless as possible.
Just IMHO, of course (especially since I'm not particularly interested
in doing the work on the upgrade process myself).

-Neil

With 7.4 I'm finding upgrading to be easier. I'll likely upgrade out
production servers to 7.4.0 when it comes out and wind up skipping 7.3
altogether.

Sure but I talking about people who are running 7.3 and are happy with
it. The reality is that for probably 95% of the people
out there , there is no reason for 7.4. When you have existing system
that works... why upgrade? That is one of the benefits
of Open Source stuff, we no longer get force into un-needed upgrade cycles.

We use PostgreSQL for everything, and I don't have any inclination to
upgrade to 7.4 except that it is 7.4. I only have two
customers that will see any real benefit from going to 7.4. The rest are
going to stay on 7.3 because they don't want:

A. The downtime
B. Unknown or unexpected problems
C. A brand new database
D. Migration costs

When you deal with the systems I do, the cost to a customer to migrate
to 7.4 would be in the minimum of 10,000-20,000 dollars.
They start to ask why were upgrading with those numbers.

That is not to say that 7.4 is not worth it from a technical sense but
for my customers, "If it ain't broke, don't fix it" is a mantra and
the reality is that 7.3 is not broke in their minds. There is
limitations pg_dump/pg_restore has some issues, having to reindex the
database
(which 7.4 doesn't fix), vacuum (which 7.4 doesn't fix) but my customers
accept them as that.

Your mileage may vary but I can only talk from my experience.

Sincerely,

Joshua D. Drake

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
The most reliable support for the most reliable Open Source database.

Maybe I've mis-read Joshua's intentions, but I got the impression that
this 7.3 maintainer would follow the patches list and backport patches
whenever possible. This way folks coding for 7.4/7.5 can stay focused on
that, but folks who can't upgrade to 7.4 for whatever reason can still
get some features / improvements.

And bug fixes but yes that is accurate.

Sincerely,

Joshua Drake

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
The most reliable support for the most reliable Open Source database.

I don't believe anyone would work against this, nor could I imagine that
anyone would think it was "a bad idea", I'm just curious as to how
possible it is to do ...

For most things probably not that possible. For things like:

Simple feature enhancements (preloading of libs)
Fixing pl/Language bugs (and making sure they still work on 7.3)
Buffer overflow fixes
Security problems (the fact that alter user/createuser with encrypted
password ' will go into a .psqlhistory file is horrendous)
pg_dump/pg_restore enhancements

Would entirely be possible.

Sincerely,

Joshua rake

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
The most reliable support for the most reliable Open Source database.

Joshua D. Drake wrote:

For most things probably not that possible. For things like:

Simple feature enhancements (preloading of libs)

How long is a piece of string? When does something stop being simple?

Fixing pl/Language bugs (and making sure they still work on 7.3)
Buffer overflow fixes

Everyone seems to agree that bugs should be fixed.

Security problems (the fact that alter user/createuser with encrypted
password ' will go into a .psqlhistory file is horrendous)

you can avoid this in the create case by using createuser -P instead of
psql. Or by using psql -c (although that might put stuff in your shell
history ;-)
Maybe there's a good case for an alteruser counterpart to createuser.

pg_dump/pg_restore enhancements

Which ones? If it is things known to be broken being fixed that comes
under the bug fix category.

cheers

andrew

On Wed, 1 Oct 2003, Joshua D. Drake wrote:

With 7.4 I'm finding upgrading to be easier. I'll likely upgrade out
production servers to 7.4.0 when it comes out and wind up skipping 7.3
altogether.

Sure but I talking about people who are running 7.3 and are happy with
it. The reality is that for probably 95% of the people
out there , there is no reason for 7.4. When you have existing system
that works... why upgrade? That is one of the benefits
of Open Source stuff, we no longer get force into un-needed upgrade cycles.

Agreed, we've been on 7.2 for a while now because it just works.
The regex substring introduced in 7.3 was a pretty cool feature, for
instance, that makes life easy.

When you deal with the systems I do, the cost to a customer to migrate
to 7.4 would be in the minimum of 10,000-20,000 dollars.
They start to ask why were upgrading with those numbers.

then maybe they would be willing to donate some small amount each ($500 or
so) to pay for backporting issues. Since mostly what I'd want on an older
version would be bug / security fixes, that $500 should go a long way
towards backporting.

That is not to say that 7.4 is not worth it from a technical sense but
for my customers, "If it ain't broke, don't fix it" is a mantra and
the reality is that 7.3 is not broke in their minds. There is
limitations pg_dump/pg_restore has some issues, having to reindex the
database
(which 7.4 doesn't fix), vacuum (which 7.4 doesn't fix) but my customers
accept them as that.

I was under the imporession that 7.4 removed the need to reindex caused by
monotonically increasing index keys, no?

Your mileage may vary but I can only talk from my experience.

Yeah, I would rather have had more back porting to 7.2 because there were
tons of little improvements form 7.2 to 7.3 I could have used while
waiting for 7.4's improved pg_dumpall to come along.

Cheers:-)

then maybe they would be willing to donate some small amount each ($500 or
so) to pay for backporting issues. Since mostly what I'd want on an older
version would be bug / security fixes, that $500 should go a long way
towards backporting.

Sure.

I was under the imporession that 7.4 removed the need to reindex caused by
monotonically increasing index keys, no?

Someone else brought that up. Maybe I am misunderstanding something but
it was my understanding that 7.4 fixes alot of
the issues but one of the issues (index bloat) although improved is not
entirely fixed and thus we would still need reindex?
Tom am I on crack?

Yeah, I would rather have had more back porting to 7.2 because there were
tons of little improvements form 7.2 to 7.3 I could have used while
waiting for 7.4's improved pg_dumpall to come along.

Well there ya go :)

Sincerely,

Joshua Drake

Cheers:-)

-- 
Command Prompt, Inc., home of Mammoth PostgreSQL - S/ODBC and S/JDBC
Postgresql support, programming shared hosting and dedicated hosting.
+1-503-222-2783 - jd@commandprompt.com - http://www.commandprompt.com
Editor-N-Chief - PostgreSQl.Org - http://www.postgresql.org