Status report

I am still going through my mailbox, trying to address all the open
patches so we can move toward beta. Of course, many of the patches I
kept need some adjustment to get applied (e.g. configuration file
location) so it is slow going.

However, we still have PITR unapplied, autovacuum unapplied, and nested
transactions don't have savepoint support, so we are still a while away
from beta. I think we should start thinking of beta as August 1 rather
than mid-July.

Plus a bunch of other minor patches :)

Chris

On Sat, 10 Jul 2004, Bruce Momjian wrote:

I am still going through my mailbox, trying to address all the open
patches so we can move toward beta. Of course, many of the patches I
kept need some adjustment to get applied (e.g. configuration file
location) so it is slow going.

However, we still have PITR unapplied, autovacuum unapplied, and nested
transactions don't have savepoint support, so we are still a while away
from beta. I think we should start thinking of beta as August 1 rather
than mid-July.

And so we fall into the hole we dig for ourselves with each release, where
the beta period takes almost as long as the development period itself ...
originally, this was all supposed to be done for June 1st, but we pushed
it back to July 1st since ppl only needed "another month" ... then, July
1st, we did the feature freeze, but because so much was left to be
applied, we decided to leave bundling first beta until mid-July ...

If six extra weeks hasn't been enough, I have little faith that adding
'yet another 2 weeks' is going to be enough either ... leave them for 7.6
then ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

My point is that it isn't the patch submitters we are waiting on
anymore. It is the backlog of patches that need review/adjustment.

---------------------------------------------------------------------------

Marc G. Fournier wrote:

On Sat, 10 Jul 2004, Bruce Momjian wrote:

I am still going through my mailbox, trying to address all the open
patches so we can move toward beta. Of course, many of the patches I
kept need some adjustment to get applied (e.g. configuration file
location) so it is slow going.

However, we still have PITR unapplied, autovacuum unapplied, and nested
transactions don't have savepoint support, so we are still a while away
from beta. I think we should start thinking of beta as August 1 rather
than mid-July.

And so we fall into the hole we dig for ourselves with each release, where
the beta period takes almost as long as the development period itself ...
originally, this was all supposed to be done for June 1st, but we pushed
it back to July 1st since ppl only needed "another month" ... then, July
1st, we did the feature freeze, but because so much was left to be
applied, we decided to leave bundling first beta until mid-July ...

If six extra weeks hasn't been enough, I have little faith that adding
'yet another 2 weeks' is going to be enough either ... leave them for 7.6
then ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Sat, 10 Jul 2004, Bruce Momjian wrote:

My point is that it isn't the patch submitters we are waiting on
anymore. It is the backlog of patches that need review/adjustment.

"Of course, many of the patches I kept need some adjustment to get applied
..." ... to me, that indicates that even after review, they will have to
go back to the submitter to be adjusted, and sent back, and reviewed, and
...

Get in what you feel comfortable adding over the next week, the rest can
wait until 7.6 ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

If you get full control of PostgreSQL, you can dictate what will happen.
Until then, I will follow the community consensus, which may or may not
match your opinion.

---------------------------------------------------------------------------

Marc G. Fournier wrote:

On Sat, 10 Jul 2004, Bruce Momjian wrote:

My point is that it isn't the patch submitters we are waiting on
anymore. It is the backlog of patches that need review/adjustment.

"Of course, many of the patches I kept need some adjustment to get applied
..." ... to me, that indicates that even after review, they will have to
go back to the submitter to be adjusted, and sent back, and reviewed, and
...

Get in what you feel comfortable adding over the next week, the rest can
wait until 7.6 ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

If you get full control of PostgreSQL, you can dictate what will happen.
Until then, I will follow the community consensus, which may or may not
match your opinion.

Um, let's take the time to get the features in, otherwise we'll be
waiting another year (roughly) to get PITR and others out to end users.

:)

Regards and best wishes,

Justin Clift

Justin Clift wrote:

Bruce Momjian wrote:

If you get full control of PostgreSQL, you can dictate what will happen.
Until then, I will follow the community consensus, which may or may not
match your opinion.

Um, let's take the time to get the features in, otherwise we'll be
waiting another year (roughly) to get PITR and others out to end users.

We can't affort not having PITR or NT in 7.5; we announced it already
everywhere. But it's not a real problem if we need some more time to
release, we always tell "we don't release according to a release plan,
but if things are mature".

Regards,
Andreas

On 7/10/2004 11:02 PM, Bruce Momjian wrote:

If you get full control of PostgreSQL, you can dictate what will happen.
Until then, I will follow the community consensus, which may or may not
match your opinion.

Marc isn't the only one who didn't like waiting for more features going
into 7.5 two months ago. I agree that it is too late now to push things
just out. But the mistake made wasn't ours.

What this repeated discussion about release plans and schedules (and we
have it every release) indicates to me is that we might miss something.
As you pointed out elsewhere, a 9-12 month development cycle just isn't
enough to get those big features done. But I think that stretching the
release cycle to two years and holding back all the smaller features as
well isn't a good idea either.

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Jan

---------------------------------------------------------------------------

Marc G. Fournier wrote:

On Sat, 10 Jul 2004, Bruce Momjian wrote:

My point is that it isn't the patch submitters we are waiting on
anymore. It is the backlog of patches that need review/adjustment.

"Of course, many of the patches I kept need some adjustment to get applied
..." ... to me, that indicates that even after review, they will have to
go back to the submitter to be adjusted, and sent back, and reviewed, and
...

Get in what you feel comfortable adding over the next week, the rest can
wait until 7.6 ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck@Yahoo.com #

Jan Wieck wrote:

On 7/10/2004 11:02 PM, Bruce Momjian wrote:

If you get full control of PostgreSQL, you can dictate what will happen.
Until then, I will follow the community consensus, which may or may not
match your opinion.

Marc isn't the only one who didn't like waiting for more features going
into 7.5 two months ago. I agree that it is too late now to push things
just out. But the mistake made wasn't ours.

What this repeated discussion about release plans and schedules (and we
have it every release) indicates to me is that we might miss something.
As you pointed out elsewhere, a 9-12 month development cycle just isn't
enough to get those big features done. But I think that stretching the
release cycle to two years and holding back all the smaller features as
well isn't a good idea either.

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Alvaro started out with ifdef's but it got too confusing and we all
agreed to just go with a normal patch. He just hits too much code.
PITR could be done that way though.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Jan Wieck wrote:

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Alvaro started out with ifdef's but it got too confusing and we all
agreed to just go with a normal patch. He just hits too much code.

I think the same would be true of almost any really large feature ---
ifdefs all over the code base are just too much of a mess.

To be honest I think that "releasing more often" isn't necessarily an
appropriate goal for the project anymore. Five or six years ago we were
doing a major (initdb-forcing) release every three or four months, and
that was appropriate at the time, but the project has matured and our
user population has changed. Look at how many people are still using
7.2 or 7.3. One major release a year may be about right now, because
you can't get people to adopt new major revs faster than that anyway.

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Jan Wieck wrote:

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Alvaro started out with ifdef's but it got too confusing and we all
agreed to just go with a normal patch. He just hits too much code.

I think the same would be true of almost any really large feature ---
ifdefs all over the code base are just too much of a mess.

To be honest I think that "releasing more often" isn't necessarily an
appropriate goal for the project anymore. Five or six years ago we were
doing a major (initdb-forcing) release every three or four months, and
that was appropriate at the time, but the project has matured and our
user population has changed. Look at how many people are still using
7.2 or 7.3. One major release a year may be about right now, because
you can't get people to adopt new major revs faster than that anyway.

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

Totally agree.

I think the only downside to a longer release cycle is that features
developed would take longer to get out to the public. Perhaps we need
to start thinking of add-ons to existing releases such as an ARC or
vacuum-delay add-on to the 7.4.X release. The patch would potentially
have to be recreated for every minor release. I would also like to see
some psql message that shows the add-ons added to an official release.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

I think the only downside to a longer release cycle is that features
developed would take longer to get out to the public. Perhaps we need
to start thinking of add-ons to existing releases such as an ARC or
vacuum-delay add-on to the 7.4.X release.

Mmm ... for people whose pain-point has to do with compatibility
testing, adding features in minor releases would turn them into major
releases, because they'd still have to wonder whether the new version
would break anything. I think our policy of "only bug fixes in stable
releases" is important to maintain, because it makes it easy for people
to upgrade within a stable release series.

Also, we do not really have the manpower to test multiple concurrently
developed versions ...

regards, tom lane

Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Tom Lane wrote:

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

I think the only downside to a longer release cycle is that features
developed would take longer to get out to the public. Perhaps we need
to start thinking of add-ons to existing releases such as an ARC or
vacuum-delay add-on to the 7.4.X release.

Mmm ... for people whose pain-point has to do with compatibility
testing, adding features in minor releases would turn them into major
releases, because they'd still have to wonder whether the new version
would break anything. I think our policy of "only bug fixes in stable
releases" is important to maintain, because it makes it easy for people
to upgrade within a stable release series.

Also, we do not really have the manpower to test multiple concurrently
developed versions ...

When I say add-ons, I am thinking of source code patches that are
avaiable as options to the standard subreleases. I agree we don't want
to change our current subrelease criteria.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Note that I'm all for a long (2 year? or even 'indefinite') development
cycle for a major release if we continue on with what has been happening
more often lately, where stuff is back patched to the last stable release
... there is alot of stuff that doesn't require a reload of the database
(or initdb) that could very easily be backpatched ...

As Jan points out, its the 'small features that are done' that we've been
griping about having to wait for, not the big ones which we know aren't
done ...

The nice thing about doing something lke that is those small features
would get a degree of testing happening in a live environment ...

On Tue, 13 Jul 2004, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Jan Wieck wrote:

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Alvaro started out with ifdef's but it got too confusing and we all
agreed to just go with a normal patch. He just hits too much code.

I think the same would be true of almost any really large feature ---
ifdefs all over the code base are just too much of a mess.

To be honest I think that "releasing more often" isn't necessarily an
appropriate goal for the project anymore. Five or six years ago we were
doing a major (initdb-forcing) release every three or four months, and
that was appropriate at the time, but the project has matured and our
user population has changed. Look at how many people are still using
7.2 or 7.3. One major release a year may be about right now, because
you can't get people to adopt new major revs faster than that anyway.

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

regards, tom lane

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

Marc G. Fournier wrote:

Note that I'm all for a long (2 year? or even 'indefinite') development
cycle for a major release if we continue on with what has been happening
more often lately, where stuff is back patched to the last stable release
... there is alot of stuff that doesn't require a reload of the database
(or initdb) that could very easily be backpatched ...

As Jan points out, its the 'small features that are done' that we've been
griping about having to wait for, not the big ones which we know aren't
done ...

The nice thing about doing something lke that is those small features
would get a degree of testing happening in a live environment ...

Of course that last sentence is the downside too --- people don't want
to have to retest their setups after a minor release upgrade.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

On Tuesday 13 July 2004 17:12, Marc G. Fournier wrote:

... there is alot of stuff that doesn't require a reload of the database
(or initdb) that could very easily be backpatched ...

As Jan points out, its the 'small features that are done' that we've been
griping about having to wait for, not the big ones which we know aren't
done ...

Split the feature out into either a patch or a module and put it in contrib
until the next major version. Let contrib hold the smaller,
non-initdb-forcing patches and such until the next major version rolls them
into the mainline. Or even a patches tree parallel to contrib. Either way,
the patch or module or whatever wouldn't be in the mainline unless the user
needed it.

Or maybe we need to rethink what a major version is. But even minor things
can force an initdb, although we're better than we have been about this.

But Tom's assertion is true. We have enough trouble getting patches rolled
out; adding parallel branches is just begging for trouble, due to our
relatively small resource size. Although, we probably have enough developers
at this point to make it happen.

Bruce I would want to be the patchmeister for the stable branch. Someone else
(with Bruce's oversight, or Tom's, or whoever) could do the patchmunging and
review for the development tree. But I want a stable hand on patches that go
into the stable tree.

The BSD's release something like that, with CURRENT, TESTING, and STABLE,
right? (I'm not a big BSD user...) The Linux kernel has parallel
development, and a gatekeeper for each stable release (2.0.x, 2.2.x, 2.4.x,
and now 2.6.x). A 2.0.x kernel release happened not too long ago, in fact.
We probably could have enough volunteers to do something like that.
Gatekeeping a stable release would not be as complex as developing the new
release, but, again, I would want an experienced hand on the last stable
release where the temptation of backporting 'features' is great. I think a
gatekeeper for 7.2.x, for example, would have very little to do except once
in a great while if and when a serious bug is found. At that point, that
gatekeeper can make a release (the current process is too tied up in people,
IMO, and that includes the RPM mechanism (which I have every right to
criticize!)).
--
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC 28772
(828)862-5554
www.pari.edu

On Tue, 2004-07-13 at 13:03 -0400, Tom Lane wrote:

Bruce Momjian <pgman@candle.pha.pa.us> writes:

Jan Wieck wrote:

I think in the future we have to force all large features, those that
probably need more than 6 months of development time, to be properly
#ifdef'd. Then it wouldn't be that big of a deal to release more often.

Alvaro started out with ifdef's but it got too confusing and we all
agreed to just go with a normal patch. He just hits too much code.

I think the same would be true of almost any really large feature ---
ifdefs all over the code base are just too much of a mess.

To be honest I think that "releasing more often" isn't necessarily an
appropriate goal for the project anymore. Five or six years ago we were
doing a major (initdb-forcing) release every three or four months, and
that was appropriate at the time, but the project has matured and our
user population has changed. Look at how many people are still using
7.2 or 7.3. One major release a year may be about right now, because
you can't get people to adopt new major revs faster than that anyway.

Of course this all ties into the pain of having to dump/reload large
databases, and maybe if we had working pg_upgrade the adoption rate
would be faster, but I'm not at all sure of that. We're playing in
a different league now. Big installations tend to want to do
significant amounts of compatibility testing before they roll out
a new database version.

regards, tom lane

It sounds like your only taking the point of view of people upgrading
previous installations. What about new installations? I'm sure there are
hundreds, if not thousands of new installations happening every week.
These people are going to grab the latest stable version without a
doubt.

I think releasing more often would result in features getting tested
much more. Then the "big installations" can see that a major feature has
been in two stable releases (even if the time period was only a year)
and feel much more comfortable in upgrading. Why would they have to
upgrade more often then necessary anyways? Assuming security exploits
are back ported of course.

--
Mike Benoit <ipso@snappymail.ca>

On Tue, 13 Jul 2004, Bruce Momjian wrote:

The nice thing about doing something lke that is those small features
would get a degree of testing happening in a live environment ...

Of course that last sentence is the downside too --- people don't want
to have to retest their setups after a minor release upgrade.

Nobody would be required to upgrade to a new minor release either ...
nobody is *require* to upgrade to any release, for that matter ...

Hell, when we have a client that comes to us with a problem, we don't
recommend upgrading unless we find something in the RELEASE NOTES to
indicate something has been fixed related to their problem ... it isn't a
automatic "well, upgrade to the latest stable first, and then we can help
you" ...

God, we still have clients using 7.2 servers, cause they've had no reason
yet to upgrade to the latest ... "it works, why upgrade?" is generally the
opinion ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664

Marc G. Fournier wrote:

On Tue, 13 Jul 2004, Bruce Momjian wrote:

The nice thing about doing something lke that is those small features
would get a degree of testing happening in a live environment ...

Of course that last sentence is the downside too --- people don't want
to have to retest their setups after a minor release upgrade.

Nobody would be required to upgrade to a new minor release either ...
nobody is *require* to upgrade to any release, for that matter ...

Hell, when we have a client that comes to us with a problem, we don't
recommend upgrading unless we find something in the RELEASE NOTES to
indicate something has been fixed related to their problem ... it isn't a
automatic "well, upgrade to the latest stable first, and then we can help
you" ...

God, we still have clients using 7.2 servers, cause they've had no reason
yet to upgrade to the latest ... "it works, why upgrade?" is generally the
opinion ...

We have always recommended upgrading to the most recent minor release,
at least as a project policy for support. Also, the upgrade is only
stop/install/restart so it is quite easy to do, and folks like the fact
they don't have to test for new functionality.

One idea would be for us to release 7.5 and 7.5.0.1, and allow 7.5.1 to
have minor new features. That way, 7.5.0.[1-9] are no new features, and
7.5.[1-9] are minor improvements against 7.5.

Of course this does require more project management, and we already
don't have enough manpower.

I was thinking of something much simpler where Jan would create an ARC
patch against 7.4.X and have it either in /contrib for 7.4.X or on our
ftp servers, or on a web site. I could create a mechanism so SELECT
version() would display Jan's add-on.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073