8.5 release timetable, again
I posted a note about a week ago which drew far less commentary than I
expected, regarding the timetable for the release of 8.5.
http://archives.postgresql.org/pgsql-hackers/2009-08/msg01256.php
Perhaps this is already being discussed on -core, but if so the
conclusions haven't been shared publicly. We started the first
CommitFest for 8.5 on July 15, 2009. I believe that there is general
consensus on a two-month cycle for CommitFests, which means that the
second one will start on September 15, 2009, and the third one on
November 15, 2009. The open question is whether there will be a
fourth or a fifth CommitFest, and what that will do to the time line
for the final release.
During the 8.4 development cycle, we started the last CommitFest on
November 1, 2008 and released 8 months later on July 1, 2009. I would
like to think that we could shave a bit of time off of that this time
around, because last time the final CommitFest took approximately 4
months. I found that to be pretty ridiculous and I think that I was
not the only one. By the end, there were not too many patches left
and most of those had been thoroughly reviewed, so there was no
obvious role for non-committers to play. There were also many patches
that went for weeks, sometimes over a month, without being updated or
bounced. I think we can improve this a lot this time around, by
applying the same rules to the last CommitFest that we did to the
first one: patches must be updated in a timely fashion. If they
can't, it's a sign that either the patch author is too busy to work on
the patch, or the revisions needed are too extensive for a single
CommitFest. Either way, we move on.
I am a lot less clear on what we can do to shorten the time we spent
in beta. In some ways, the time we spent in beta for 8.4 seems not to
have been long enough. Several critical bugs introduced by the
infrastructure-changes-for-recovery patch were not fixed until just
before release, and a number of planner and other significant bugs
have been found since the release and will be fixed in 8.4.1. On the
other hand, from my point of view, there was nothing to do during
beta. Most of the open items required first and foremost a decision
about the best way to fix them, and those decisions really need to be
made by (or at least with the consent of) the committers. If we have
a substantial open items list again for 8.5, we need to find a way to
formalize the handling of that list so that it can be dealt with
efficiently and the work distributed among as many people as are
willing and able to help. Writing release notes also seemed to take a
lot of time, perhaps partly because we didn't know until the very end
whether certain large patches were going to be committed.
All of the above having been said, I think it's too much to hope that
the beta/release cycle is going to get drastically shorter than it was
for 8.4. So I think if we decide on three CommitFests, the timetable
will end up going something like this:
2009-11-15 Last CommitFest Begins
2009-12-15 Last CommitFest Ends
2010-01-01 Alpha
2010-03-01 Beta
2010-05-01 Release
This schedule assumes that instead of having 8 months between
start-of-last-CommitFest and release, we'll have five and a half.
Assuming that we can avoid the temptation to let the last CommitFest
drag on and on and on, that seems achievable.
If we go with four CommitFests, we'll probably end up release 45-60
days later, in mid June or start of July. If we go with five
CommitFests, we'll probably be looking at September.
Personally, I favor the more aggressive schedule of just three
CommitFests, because if something goes wrong and the schedule does
slip, we should still be able to get the release out the door next
year by the same time we did this year. If things go as planned,
we'll have the release out the door in time for PGCon. If things go
truly excellently and we manage to get the release out before 1-May,
then we have that much more time for 8.6 development. On the other
hand, if we plan for four Commitfests, we'll be looking at releasing
the same time next year that we did this year *if* everything goes as
planned. If anything slips, we'll release even later - and that also
gets into the summer time, when more people are away.
But, some other decision that has consensus is fine too. The most
important thing is to MAKE a decision before the passage of time makes
one for us.
...Robert
Robert Haas <robertmhaas@gmail.com> writes:
I posted a note about a week ago which drew far less commentary than I
expected, regarding the timetable for the release of 8.5.
http://archives.postgresql.org/pgsql-hackers/2009-08/msg01256.php
Perhaps this is already being discussed on -core, but if so the
conclusions haven't been shared publicly.
Core hasn't discussed 8.5 schedule since the discussions that Peter
summarized in the message you cited above. I share your concern that
"release in time for PGCon" isn't very realistic if we don't get more
aggressive about schedule. On the other hand, we didn't get all that
much done in this fest. If we cut back to a three-fest schedule
I think we may succeed in releasing 8.5 early, but only because there
is nothing interesting in it :-(. Dunno where the right balance is.
regards, tom lane
On Sun, Aug 23, 2009 at 1:57 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
I posted a note about a week ago which drew far less commentary than I
expected, regarding the timetable for the release of 8.5.http://archives.postgresql.org/pgsql-hackers/2009-08/msg01256.php
Perhaps this is already being discussed on -core, but if so the
conclusions haven't been shared publicly.Core hasn't discussed 8.5 schedule since the discussions that Peter
summarized in the message you cited above. I share your concern that
"release in time for PGCon" isn't very realistic if we don't get more
aggressive about schedule. On the other hand, we didn't get all that
much done in this fest. If we cut back to a three-fest schedule
I think we may succeed in releasing 8.5 early, but only because there
is nothing interesting in it :-(. Dunno where the right balance is.
Here is my thinking on that point. We have several major features
underway for the September CommitFest, including at least Hot Standby
(Simon Riggs), Streaming Replication (Fujii Masao), and Index-Only
Scans (Heikki). At the July CommitFest, none of these patches were in
a state where they could be seriously reviewed. Simon Riggs and Fujii
Masao have both committed to produce reviewable versions by 9/15, and
it looks like Heikki will hit that date too, if not beat it.
I am assuming that at least Hot Standby and Streaming Replication will
likely require two CommitFests to go from the point where they are
seriously reviewable to actual commit. So if they hit the 9/15 date,
they should make 8.5 even with just three CommitFests. If they don't
hit the 9/15 date, then a 3-CommitFest cycle will probably be too
short for them to make it in. But if we schedule a fourth CommitFest
in January in the hopes of seeing one of those patches committed, then
ISTM we're basically speculating that the patch authors will not hit
the 9/15 date but that they will hit an 11/15 date.
To me, that's an entirely arbitrary and unfounded speculation. On the
one hand, both patch authors have said they will hit 9/15, so why
should we second-guess them? On the other hand, neither of those
patches seems to have made a great deal of progress in the last 7
months, so it's unclear that tacking a few more months onto the
schedule will help anything. Furthermore, even if we're right that
four CommitFests is the right number to land one of those patches
(rather than 3 or 5 or 6 or 7), we're then talking about committing a
major feature in the very last CommitFest for this release. We tried
that for 8.4 and it wasn't a stunning success.
To some degree, what this boils down to is that you can have
time-based releases or feature-based releases, but not both. And the
problem with feature-based releases in a community environment is that
there is no guarantee that patches will be delivered when promised.
None of the people developing these major features work for the
community and we do not have the ability to control any of their
schedules. So saying that we're going to wait for them to be ready is
potentially saying that we're going to wait forever. No one is
proposing that, but we are sort of trying to read the tea leaves and
try to guess when they might be ready and tailor the schedule to it.
To me, it seems to make more sense to just decide we're going to ship
the release on a time line that works for the project overall. If we
get some new features in, good. If they slip a bit past the deadline,
well, at least that should hopefully mean that they get committed
right at the beginning of the 8.6 cycle. If they slip WAY past the
deadline, bummer, but at least we didn't hold up the release to no
benefit. I don't think there's anything wrong with having a release
that has many small improvements but no major new features, and I
think that is the worst that will happen.
...Robert
On sön, 2009-08-23 at 09:37 -0400, Robert Haas wrote:
To some degree, what this boils down to is that you can have
time-based releases or feature-based releases, but not both.
Sure. But some people are trying to introduce another subvariant: The
conference-circuit-based releases. ;-) It sounds attractive, but it
shouldn't trump all other concerns. Consider this instead: Nothing to
do during beta? Write your conference slides! ;-)
I suggest going with four commit fests. Three is too short. We already
started the first one early, which didn't give those involved in the
release any time to prepare some patches for it. So with three fests
you'd only give the major developers 8 weeks to code something for a
yearly release.
On Mon, Aug 24, 2009 at 7:39 AM, Peter Eisentraut<peter_e@gmx.net> wrote:
On sön, 2009-08-23 at 09:37 -0400, Robert Haas wrote:
To some degree, what this boils down to is that you can have
time-based releases or feature-based releases, but not both.Sure. But some people are trying to introduce another subvariant: The
conference-circuit-based releases. ;-) It sounds attractive, but it
shouldn't trump all other concerns. Consider this instead: Nothing to
do during beta? Write your conference slides! ;-)
Oh, gee, what am I speaking on? :-)
The main thing that I (can't speak for anyone else) like about getting
a release out in time for PGCon is that it happens at about the same
time every year, and I kind of like the idea of shooting for a yearly
cycle. I'd be just as happy to release every year on Christmas day,
but it's harder to get people to bundle a release around then.
I suggest going with four commit fests. Three is too short. We already
started the first one early, which didn't give those involved in the
release any time to prepare some patches for it. So with three fests
you'd only give the major developers 8 weeks to code something for a
yearly release.
Well, that's a good point, but 12 weeks out of a 14-month release
cycle is only marginally better than 8 weeks out of a 12-month release
cycle. If we really want to give people more time to write patches,
we need to figure out how to speed up the time from the end of the
last CommitFest until release.
One thing that I had thought of proposing is that we branch the tree
when we go to 8.5-beta and hold the first 8.6 CommitFest at that time
- or, failing that, that we hold a DontCommitFest at that time, where
we review all of the patches just as we would for a regular
CommitFest, but without the committing part. At least for 8.4, it
didn't seem like much was happening during beta, at least not much
that was discussed on -hackers or could be distributed across the
community.
I actually don't think that's the ideal solution, though. It's just
papering around the problem that the release takes too long and the
burden falls on too few people. But we haven't yet come up with any
good idea to address that problem.
At any rate, I'm OK with 4 CommitFests if that is the consensus, but
so far we only have 3 people weighing in on this thread, which is not
a consensus for anything.
...Robert
peter_e@gmx.net (Peter Eisentraut) writes:
I suggest going with four commit fests. Three is too short. We already
started the first one early, which didn't give those involved in the
release any time to prepare some patches for it. So with three fests
you'd only give the major developers 8 weeks to code something for a
yearly release.
Partial counter-argument...
A large portion of the patches in CommitFest #1 represented items that
had been deferred from 8.4. So...
a) Many of these patches came in with ~6 months of preparation time
b) People were always free to start work earlier than CommitFest #1
c) If something requires a *lot* of work, then it may be that it
gets deferred so that it comes in as part of CommitFest #1 for
8.6, with the very same characteristics as in a)...
I do agree that trying to force coordination with a specific conference
in Ottawa seems like a very peculiar sort of forced scheduling.
--
select 'cbbrowne' || '@' || 'ca.afilias.info';
Christopher Browne
"Bother," said Pooh, "Eeyore, ready two photon torpedoes and lock
phasers on the Heffalump, Piglet, meet me in transporter room three"
Christopher Browne <cbbrowne@ca.afilias.info> writes:
I do agree that trying to force coordination with a specific conference
in Ottawa seems like a very peculiar sort of forced scheduling.
Well, PGCon is just a convenient concrete target. The real point here
is that we're trying to get the release cycle to end with a release in
the spring. We've grown tired of releasing in the fall and invariably
finding the date slipping into holiday season, so we'd like to try
releasing near the other equinox. (Neither solstice is good, since
you are up against either holidays or vacations taking away the time
of the people who need to be doing the work.)
regards, tom lane
On Mon, Aug 24, 2009 at 5:55 PM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Christopher Browne <cbbrowne@ca.afilias.info> writes:
I do agree that trying to force coordination with a specific conference
in Ottawa seems like a very peculiar sort of forced scheduling.Well, PGCon is just a convenient concrete target. The real point here
is that we're trying to get the release cycle to end with a release in
the spring.
Yeah, conference-based releases is just a proxy for time-based
releases. It's nice to have something to be happy about at the
conference too. And it's a convenient time to start talking about the
next release when you're all face-to-face.
All,
Yeah, conference-based releases is just a proxy for time-based
releases. It's nice to have something to be happy about at the
conference too. And it's a convenient time to start talking about the
next release when you're all face-to-face.
On the one hand:
I'd say that we go for the 3-CF release. I think we need to prove that
we can do a time-based release once before a lot of people on this list
will believe in it.
If we do 4 CFs, we're in danger of still being in beta in late May ...
and once conference and vacation season start, things get a lot slower.
Mind you, it's possible that we can shorten the final CF and beta this
time, but I wouldn't want to count on it.
If we *can* shorten them, then 8.6 can have 4CFs. But we won't know
until after we've done it.
On the other hand:
I think if we do another release without Standby/replication, we'll
start to lose a lot of users. People are waiting on that, and a lot of
folks were expecting it in 8.4.
Therefore: I think, 3CFs, but we go all-out to get Standby/Replication
into 8.5 in the next month. So, every committer/major hacker on this
list should pitch in to get those features done.
So, is there someone here who could be helping with HS/SR and isn't?
Why not?
--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com
On Mon, Aug 24, 2009 at 1:48 PM, Josh Berkus<josh@agliodbs.com> wrote:
Therefore: I think, 3CFs, but we go all-out to get Standby/Replication
into 8.5 in the next month. So, every committer/major hacker on this
list should pitch in to get those features done.So, is there someone here who could be helping with HS/SR and isn't?
Why not?
Unfortunately, neither Simon nor Fujii Masao is publishing a git
repository of their work or regular updates to their patch set.
Therefore, no one else can contribute code, and in fact no one else
can even provide a review, because there is nothing to look at. I
don't think that there would be any problem getting these patches
reviewed/committed outside of the regular CommitFest cycle, but
there's simply no place to start.
I have volunteered to work on Hot Standby and have merged it up to CVS
HEAD several times, and cleaned out a bit of other cruft, but the
scope of the task appears to exceed the time I have available. It
seems to me that other people could clone my git repo (or that branch)
and submit patches against it (or ask me to pull from a branch that
they publish), but will Simon incorporate those patches (assuming that
they don't suck) into his next version, or ignore them? It's not
really clear.
As I've said before, I am presently of the opinion that Streaming
Replication has little chance of making it into 8.5. This opinion is
vulnerable to contrary evidence, like a new version of the patch
showing up that shows massive progress. But the patch was bounced
from CF 2009-07 for a whole series of architectural problems which
have to be addressed before we can even get to implementation details,
bugs, documentation, etc. Hot Standby is in better shape but amount
of code cleanup needed is substantial and there is also quite a bit of
'git diff master | grep XXX' that needs to be gone through.
...Robert
Josh Berkus <josh@agliodbs.com> writes:
So, is there someone here who could be helping with HS/SR and isn't?
Why not?
You mean, other than Simon's hands-off attitude?
regards, tom lane
Josh Berkus wrote:
All,
Yeah, conference-based releases is just a proxy for time-based
releases. It's nice to have something to be happy about at the
conference too. And it's a convenient time to start talking about the
next release when you're all face-to-face.On the one hand:
I'd say that we go for the 3-CF release. I think we need to prove that
we can do a time-based release once before a lot of people on this list
will believe in it.If we do 4 CFs, we're in danger of still being in beta in late May ...
and once conference and vacation season start, things get a lot slower.
Mind you, it's possible that we can shorten the final CF and beta this
time, but I wouldn't want to count on it.If we *can* shorten them, then 8.6 can have 4CFs. But we won't know
until after we've done it.On the other hand:
I think if we do another release without Standby/replication, we'll
start to lose a lot of users. People are waiting on that, and a lot of
folks were expecting it in 8.4.
That is a slightly alarmist. Who are we going to lose these users to?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
Robert Haas wrote:
As I've said before, I am presently of the opinion that Streaming
Replication has little chance of making it into 8.5. This opinion is
vulnerable to contrary evidence, like a new version of the patch
showing up that shows massive progress. But the patch was bounced
from CF 2009-07 for a whole series of architectural problems which
have to be addressed before we can even get to implementation details,
bugs, documentation, etc. Hot Standby is in better shape but amount
of code cleanup needed is substantial and there is also quite a bit of
'git diff master | grep XXX' that needs to be gone through.
I agree. I think it is unlikely we will have anything ready to commit
for Streaming Replication or Hot Standby for the next commit-fest in
mid-September, and if we go for a 3-CF (commit fest) release, that gives
us only one final CF to get those features accepted, again unlikely. We
are either going to need to go to a 4-CF release, change the way we are
developing these patches, or both to get either in 8.5.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
On Mon, Aug 24, 2009 at 4:56 PM, Bruce Momjian<bruce@momjian.us> wrote:
Robert Haas wrote:
As I've said before, I am presently of the opinion that Streaming
Replication has little chance of making it into 8.5. This opinion is
vulnerable to contrary evidence, like a new version of the patch
showing up that shows massive progress. But the patch was bounced
from CF 2009-07 for a whole series of architectural problems which
have to be addressed before we can even get to implementation details,
bugs, documentation, etc. Hot Standby is in better shape but amount
of code cleanup needed is substantial and there is also quite a bit of
'git diff master | grep XXX' that needs to be gone through.I agree. I think it is unlikely we will have anything ready to commit
for Streaming Replication or Hot Standby for the next commit-fest in
mid-September, and if we go for a 3-CF (commit fest) release, that gives
us only one final CF to get those features accepted, again unlikely. We
are either going to need to go to a 4-CF release, change the way we are
developing these patches, or both to get either in 8.5.
I don't think a 4-CF release is going to help. It's just going to be
2 more months before everything else that has been done gets released.
Call me a pessimist if you will, but zero times an arbitrary number
of CommitFests is still zero.
The only solution here is to get more people working on these patches.
I have volunteered to work on HS and would also be willing to work on
SR. Work can be reviewing or actual code. But I cannot work on a
patch I cannot see, and neither can anyone else.
...Robert
On Monday 24 August 2009 3:51:31 pm Bruce Momjian wrote:
folks were expecting it in 8.4.
That is a slightly alarmist. Who are we going to lose these users to?
the insane asylum?
Robert Haas wrote:
On Mon, Aug 24, 2009 at 4:56 PM, Bruce Momjian<bruce@momjian.us> wrote:
Robert Haas wrote:
As I've said before, I am presently of the opinion that Streaming
Replication has little chance of making it into 8.5. ?This opinion is
vulnerable to contrary evidence, like a new version of the patch
showing up that shows massive progress. ?But the patch was bounced
from CF 2009-07 for a whole series of architectural problems which
have to be addressed before we can even get to implementation details,
bugs, documentation, etc. ?Hot Standby is in better shape but amount
of code cleanup needed is substantial and there is also quite a bit of
'git diff master | grep XXX' that needs to be gone through.I agree. ?I think it is unlikely we will have anything ready to commit
for Streaming Replication or Hot Standby for the next commit-fest in
mid-September, and if we go for a 3-CF (commit fest) release, that gives
us only one final CF to get those features accepted, again unlikely. ?We
are either going to need to go to a 4-CF release, change the way we are
developing these patches, or both to get either in 8.5.I don't think a 4-CF release is going to help. It's just going to be
2 more months before everything else that has been done gets released.
Call me a pessimist if you will, but zero times an arbitrary number
of CommitFests is still zero.The only solution here is to get more people working on these patches.
I have volunteered to work on HS and would also be willing to work on
SR. Work can be reviewing or actual code. But I cannot work on a
patch I cannot see, and neither can anyone else.
Agreed, so we fall back to "change the way we are developing these
patches". I am hesistant to jump into managing these patches until they
get their shot by their original authors in the September CF, but
managing them starting in mid-October will probably be too late for them
to get into 8.5.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ If your life is a hard drive, Christ can be your backup. +
On Mon, Aug 24, 2009 at 04:51:31PM -0400, Bruce Momjian wrote:
Josh Berkus wrote:
All,
Yeah, conference-based releases is just a proxy for time-based
releases. It's nice to have something to be happy about at the
conference too. And it's a convenient time to start talking
about the next release when you're all face-to-face.On the one hand:
I'd say that we go for the 3-CF release. I think we need to prove
that we can do a time-based release once before a lot of people on
this list will believe in it.If we do 4 CFs, we're in danger of still being in beta in late May
... and once conference and vacation season start, things get a
lot slower. Mind you, it's possible that we can shorten the final
CF and beta this time, but I wouldn't want to count on it.If we *can* shorten them, then 8.6 can have 4CFs. But we won't
know until after we've done it.On the other hand:
I think if we do another release without Standby/replication,
we'll start to lose a lot of users. People are waiting on that,
and a lot of folks were expecting it in 8.4.That is a slightly alarmist. Who are we going to lose these users
to?
Sadly, to one of the MySQL forks. This is one of those cases (cf. the
current thread on -advocacy) where pointy-hair-friendliness can really
help or hurt us.
Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
I think if we do another release without Standby/replication, we'll
start to lose a lot of users. People are waiting on that, and a lot of
folks were expecting it in 8.4.That is a slightly alarmist. Who are we going to lose these users to?
Drizzle. MySQL forks. CouchDB. Any database which has replication
which you don't need a professional DBA to understand. Whether or not
it works.
--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com
Josh Berkus <josh@agliodbs.com> writes:
That is a slightly alarmist. Who are we going to lose these users to?
Drizzle. MySQL forks. CouchDB. Any database which has replication
which you don't need a professional DBA to understand. Whether or not
it works.
You haven't explained why we'd lose such folk next year when we haven't
lost them already. MySQL has had replication (or at least has checked
off the bullet point ;-)) for years. I'd be seriously surprised if any
of the forks will offer significantly better replication than is there
now, so the competitive situation is not changing in that regard.
It is true that we're missing a chance to pull some folks away while
the situation on that side of the fence is so messy. But I don't see
our situation getting worse because of that, just not getting better.
regards, tom lane
On Mon, Aug 24, 2009 at 08:02:31PM -0400, Tom Lane wrote:
Josh Berkus <josh@agliodbs.com> writes:
That is a slightly alarmist. Who are we going to lose these
users to?Drizzle. MySQL forks. CouchDB. Any database which has
replication which you don't need a professional DBA to understand.
Whether or not it works.You haven't explained why we'd lose such folk next year when we
haven't lost them already. MySQL has had replication (or at least
has checked off the bullet point ;-)) for years. I'd be seriously
surprised if any of the forks will offer significantly better
replication than is there now, so the competitive situation is not
changing in that regard.It is true that we're missing a chance to pull some folks away while
the situation on that side of the fence is so messy. But I don't
see our situation getting worse because of that, just not getting
better.
"Not getting better," isn't a situation to be dismissed lightly. In
FLOSS, as I've seen it, a project whose adoption isn't growing is
dying.
Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
