Lessons from commit fest

Bruce Momjian wrote:

There has been talk of the lessons we learned during this commit fest,
but exactly what lessons did we learn? I am not clear on that so I
assume others are not as well. What are we going to do differently
during the next commit fest?

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

Some of it is already there.

Something else we learned is that the archives are central (well, we
already knew that, but I don't think we had ever given them so broad
use), and we've been making changes to them so that they are more useful
to reviewers. Further changes are still needed on them, of course, to
address the remaining problems.

Lastly, I would say that pushing submitters to enter their sent patches
into the Wiki worked -- we need to ensure that they keep doing it.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Apr 15, 2008 at 2:45 AM, Alvaro Herrera wrote:

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

In a wiki context, this sort of thing has got "Template" written all
over it. I've done a little editing on Wikipedia, so I've got some
idea about how to make wiki Templates work. I'm not claiming to be an
expert, but if nobody else has a particular yen for it, I can have a
go at setting up some simple templates to make it easier to add a
"patch" or a "review" in a structured way.

Lastly, I would say that pushing submitters to enter their sent patches
into the Wiki worked -- we need to ensure that they keep doing it.

+1. Although I wouldn't say that there has yet been a "push" for
submitters to enter their patches into the wiki =) I've started
adding my own patches to the wiki recently. The only thing about the
process that sucks is that I need a URL linking to the message in the
archives. I naturally want to add my patch to the wiki immediately
after sending my email to -patches, and it takes some material
interval of time for messages to show up on the archive. My solution
was to just pull the message ID out of the headers in gmail and fudge
the URL. So the URL I add to the wiki is actually borked until the
archives refresh themselves, which is less than awesome ...

Apart from the archive linkage thing, I found the process of queueing
my own patches smooth, straightforward and satisfying. I would
recommend it to other submitters, if for no other reason than to
reduce the amount of drudge work the core team has to do to keep
things in order.

Cheers,
BJ

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: http://getfiregpg.org

iD8DBQFIA5Jl5YBsbHkuyV0RAnL0AJ97+ZdXr71Xu6wMSlVGSvy1t4WqbgCgz58X
GHMaO7j4g8+WTmcNx2SKBnA=
=xPJ3
-----END PGP SIGNATURE-----

Brendan Jurd escribi�:

On Tue, Apr 15, 2008 at 2:45 AM, Alvaro Herrera wrote:

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

In a wiki context, this sort of thing has got "Template" written all
over it. I've done a little editing on Wikipedia, so I've got some
idea about how to make wiki Templates work. I'm not claiming to be an
expert, but if nobody else has a particular yen for it, I can have a
go at setting up some simple templates to make it easier to add a
"patch" or a "review" in a structured way.

Please have a go at it in a test page. The main principle we need is
that the thing must be as easy as possible to edit. (FWIW if you can
come up with a better markup than what we currently use, we'd welcome
the advise.)

Lastly, I would say that pushing submitters to enter their sent patches
into the Wiki worked -- we need to ensure that they keep doing it.

+1. Although I wouldn't say that there has yet been a "push" for
submitters to enter their patches into the wiki =)

Well, I pushed some authors via private email. Others did not seem to
need any pushing :-)

I've started adding my own patches to the wiki recently. The only
thing about the process that sucks is that I need a URL linking to the
message in the archives. I naturally want to add my patch to the wiki
immediately after sending my email to -patches, and it takes some
material interval of time for messages to show up on the archive. My
solution was to just pull the message ID out of the headers in gmail
and fudge the URL.

Right, that's the way I use them too. Sorry we don't have anything
better :-( The archives are refreshed every 10 minutes, so it's not
like you need to wait for a week, either.

Of course, I've configured my email client so that Message-Ids are shown
all over the place, so I don't have to mess around clicking stuff.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Brendan Jurd wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Apr 15, 2008 at 2:45 AM, Alvaro Herrera wrote:

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

In a wiki context, this sort of thing has got "Template" written all
over it. I've done a little editing on Wikipedia, so I've got some
idea about how to make wiki Templates work. I'm not claiming to be an
expert, but if nobody else has a particular yen for it, I can have a
go at setting up some simple templates to make it easier to add a
"patch" or a "review" in a structured way.

One problem I saw is that people commenting in the wiki sometimes didn't
leave their names. It would be nice if that could be improved.

Lastly, I would say that pushing submitters to enter their sent patches
into the Wiki worked -- we need to ensure that they keep doing it.

+1. Although I wouldn't say that there has yet been a "push" for
submitters to enter their patches into the wiki =) I've started
adding my own patches to the wiki recently. The only thing about the
process that sucks is that I need a URL linking to the message in the
archives. I naturally want to add my patch to the wiki immediately
after sending my email to -patches, and it takes some material
interval of time for messages to show up on the archive. My solution
was to just pull the message ID out of the headers in gmail and fudge
the URL. So the URL I add to the wiki is actually borked until the
archives refresh themselves, which is less than awesome ...

Yea, I have to wait to add URLs to the TODO list too, but as you
mentioned I can now use message-id URLs, though I try to avoid them
beccause they aren't as attractive --- doesn't matter for the wiki.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Alvaro Herrera <alvherre@commandprompt.com> writes:

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

I think it'd be easy to go overboard there. One thing we learned from
Bruce's page is that a display with ten or more lines per work item is
not very helpful ... you start wishing you had a summary, and then the
whole design cycle repeats.

We should not try to make the wiki page be a substitute for reading the
linked-to discussions. (This is one of the reasons that dropped links
in the archives, such as the month-end problem, are so nasty.)

One idea is to make more effective use of horizontal space than we were
doing with the current wiki-page markup. I'd have no objection to
including the author's name and a status indicator if it all fit on the
same line as the patch title/link.

regards, tom lane

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, Apr 15, 2008 at 4:12 AM, Tom Lane wrote:

Alvaro Herrera writes:

As far as the Wiki page is concerned, it would be good to make sure the
entries have a bit more info than just a header line -- things such as
"author", who reviewed and what did the reviewer say about it.

We should not try to make the wiki page be a substitute for reading the
linked-to discussions. (This is one of the reasons that dropped links
in the archives, such as the month-end problem, are so nasty.)

One idea is to make more effective use of horizontal space than we were
doing with the current wiki-page markup. I'd have no objection to
including the author's name and a status indicator if it all fit on the
same line as the patch title/link.

When you say "horizontal space", I start thinking in terms of a table,
like the ancestor of the current wiki commitfest queue, ye olde
http://developer.postgresql.org/index.php/Todo:PatchStatus

However, the table format doesn't provide well for brief review
comments, such as we currently have for the 64bit pass-by-value patch
in the May CommitFest page.

Personally I think the review addenda are nice. As Tom says, it
shouldn't be a replacement for actually reading the thread, but it
helps to get people up to speed on the status of the patch. I think
it's a useful primer for those who may be interested in looking at the
patch in more detail.

Anyway, take a look at http://wiki.postgresql.org/wiki/Template:Patch,
and http://wiki.postgresql.org/wiki/User_talk:Direvus for an example
of how it is used. This is my first stab at writing a patch queue
entry template. It uses a similar structure to what's already on the
CommitFest pages.

To make this work, all a patch submitter needs to do is go to the
relevant CommitFest page and add {{patch|||}} to the Pending Patches
section.

If you guys find this useful, I'd be happy to add a "review" template
in a similar vein.

Cheers,
BJ
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.7 (GNU/Linux)
Comment: http://getfiregpg.org

iD8DBQFIA6eX5YBsbHkuyV0RAkzNAKDEbtZZOyUI1olsErxgp1o39dH3VQCfcBQN
b25k/nEy7qupYsthhPtU7eQ=
=uu+E
-----END PGP SIGNATURE-----

Bruce Momjian <bruce@momjian.us> writes:

There has been talk of the lessons we learned during this commit fest,
but exactly what lessons did we learn?

Actually, the *main* lesson we learned was "don't start with a
2000-email inbox". There were a couple of reasons that the queue
was so forbidding:

1. We were in feature freeze for 11 months and consequently built up
a pretty sizable backlog of stuff that had been postponed to 8.4.
We have to avoid ever doing that again. We've already made some
process changes to try to avoid getting stuck that way, and we have
to be willing to change some more if the current plan doesn't work.
But that wasn't a lesson of the commit fest, we already knew it was
broken :-(. This was just inevitable pain from our poor management
of the last release cycle.

2. A whole lot of the 2000 emails were not actually about reviewable
patches. I'm willing to take most of the blame here --- I pushed
Bruce to publish the list before he'd finished doing as much clean-up
as he wanted, and I also encouraged him to leave in some long design
discussion threads that seemed to me to warrant more discussion.
(And part of the reason I thought so was that I'd deliberately ignored
those same threads when they were active, because I was busy trying
to get 8.3 out the door; so again this was partly delayed pain from
the 8.3 mess.)

In hindsight we didn't get very much design discussion done during
the fest, and I think it's unlikely to happen in future either.
We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

A smaller lesson was that you can't start fest without a queue of
ready-to-work-on patches. We seem to be evolving towards a plan
where stuff gets dumped onto the wiki page more or less immediately
as it comes in. That should take care of that problem, though I'd
still like to see someone accept responsibility for making sure
patches get listed whether or not their author does it.

Other lessons that were already brought up:
* Bruce's mbox page was not a good status tool, despite his efforts
to improve it;
* If Bruce and I are the only ones doing anything, it's gonna be slow.

regards, tom lane

Tom Lane wrote:

A smaller lesson was that you can't start fest without a queue of
ready-to-work-on patches. We seem to be evolving towards a plan
where stuff gets dumped onto the wiki page more or less immediately
as it comes in. That should take care of that problem, though I'd
still like to see someone accept responsibility for making sure
patches get listed whether or not their author does it.

I'm on it. If someone wants to act as backup, he or she is more than
welcome.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tom Lane wrote:

the fest, and I think it's unlikely to happen in future either.
We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

We have discouraged design discussions during feature freeze and beta ---
do we want to change that?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:

A smaller lesson was that you can't start fest without a queue of
ready-to-work-on patches. We seem to be evolving towards a plan
where stuff gets dumped onto the wiki page more or less immediately
as it comes in. That should take care of that problem, though I'd
still like to see someone accept responsibility for making sure
patches get listed whether or not their author does it.

Other lessons that were already brought up:
* Bruce's mbox page was not a good status tool, despite his efforts
to improve it;
* If Bruce and I are the only ones doing anything, it's gonna be slow.

Even after the wiki was setup there was still limited involvement in
patch application except for me and Tom. The wiki is going to allow
people to see more easily what is outstanding, but it doesn't seem to
have translated into more people involved in finishing the commit fest.

Humorously, it is like televising a football game --- more people watch,
but it doesn't help the players on the field. :-(

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian wrote:

Tom Lane wrote:

the fest, and I think it's unlikely to happen in future either.
We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

We have discouraged design discussions during feature freeze and beta ---
do we want to change that?

In fact, discussions were discouraged during the March commitfest too.

Perhaps this is a good idea -- discussions on new development should
be deferred until the end of the commitfest, if one is in progress.
We'll see what happens on the May commitfest, but my guess is that it
will be a lot shorter than the March one. If it takes 1.5-2 weeks, it's
not so bad, and it means people is eager to get the current patches done
as quickly as possible so that discussions on items they are interested
in can continue.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:

Bruce Momjian wrote:

Tom Lane wrote:

the fest, and I think it's unlikely to happen in future either.
We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

We have discouraged design discussions during feature freeze and beta ---
do we want to change that?

In fact, discussions were discouraged during the March commitfest too.

Perhaps this is a good idea -- discussions on new development should
be deferred until the end of the commitfest, if one is in progress.
We'll see what happens on the May commitfest, but my guess is that it
will be a lot shorter than the March one. If it takes 1.5-2 weeks, it's
not so bad, and it means people is eager to get the current patches done
as quickly as possible so that discussions on items they are interested
in can continue.

If you do that then the discussions bunch up, which is part of the
reason we had 2k emails for the March commit fest. Tom is saying not to
delay those discussions.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

We have discouraged design discussions during feature freeze and beta ---
do we want to change that?

No, changing that wasn't what I meant to suggest. My point here is that
we'd dropped a number of big mushy discussions into the queue with the
idea that they'd be re-opened during commit fest, but new discussion
did not happen to any useful extent. Conclusion: don't bother putting
anything less concrete than a patch on the fest queue.

regards, tom lane

Bruce Momjian wrote:

Alvaro Herrera wrote:

Perhaps this is a good idea -- discussions on new development should
be deferred until the end of the commitfest, if one is in progress.
We'll see what happens on the May commitfest, but my guess is that it
will be a lot shorter than the March one. If it takes 1.5-2 weeks, it's
not so bad, and it means people is eager to get the current patches done
as quickly as possible so that discussions on items they are interested
in can continue.

If you do that then the discussions bunch up, which is part of the
reason we had 2k emails for the March commit fest. Tom is saying not to
delay those discussions.

The problem here was that the discussions bunched up during a full year.
Having them bunch up for a couple of weeks is not so bad.

And the problem with those discussions continuing is that the developers
participating in them are not working in the items in the queue. We
want to encourage them to review the things already done, not to get
distracted on new stuff.

Of course, it is cat-herding, so it's not going to work 100%.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera <alvherre@commandprompt.com> writes:

Perhaps this is a good idea -- discussions on new development should
be deferred until the end of the commitfest, if one is in progress.

Well, that was part of the original idea of a commit fest, on the theory
that everyone should be helping review/commit instead. But not that
many people seem to have bought into it ...

We'll see what happens on the May commitfest, but my guess is that it
will be a lot shorter than the March one. If it takes 1.5-2 weeks, it's
not so bad, and it means people is eager to get the current patches done
as quickly as possible so that discussions on items they are interested
in can continue.

Yeah. This first one was going to be an aberration from the get-go,
just because of the backlog. I don't think we should draw too many
conclusions from it about how future ones will go. But they'd *better*
be shorter.

regards, tom lane

Tom Lane wrote:

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

We should probably try to limit the focus of fests to consider only
actual patches, with design discussions handled "live" during normal
development, the way they've always been in the past.

We have discouraged design discussions during feature freeze and beta ---
do we want to change that?

No, changing that wasn't what I meant to suggest. My point here is that
we'd dropped a number of big mushy discussions into the queue with the
idea that they'd be re-opened during commit fest, but new discussion
did not happen to any useful extent. Conclusion: don't bother putting
anything less concrete than a patch on the fest queue.

So when/how do those discussions get resolved?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

"Tom Lane" <tgl@sss.pgh.pa.us> writes:

Other lessons that were already brought up:
* Bruce's mbox page was not a good status tool, despite his efforts
to improve it;
* If Bruce and I are the only ones doing anything, it's gonna be slow.

How did you feel about the idea of having a Tom-free commitfest for May? You
would get to sit back and comment on other people's attempt to review patches,
just shouting if they seem to be headed in the wrong direction. And of course
work on your own ideas you've probably been itching to do since before 8.3
feature freeze.

I assume you realize it's not that I don't appreciate having you doing all
this work but I think it would be a good exercise for us to go through do
once. (And you certainly deserve a break!)

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's 24x7 Postgres support!

On Mon, 14 Apr 2008 21:12:59 +0100
Gregory Stark <stark@enterprisedb.com> wrote:

I assume you realize it's not that I don't appreciate having you
doing all this work but I think it would be a good exercise for us to
go through do once. (And you certainly deserve a break!)

Although I applaud the sentiment I think a more reasonable approach
would be (if Tom wanted) to have Tom pick 3-5 patches (or whatever) that
are his deal. That's it. No extra bubble activities for you. Except for
Buddha level oversight the rest falls on the rest of the community.

It isn't like we don't have at least 6 major contributors that can do
patch review... Alvaro, Greg, Neil, AndrewD, Heikki, and Magnus ... Not
to mention a slew of others who can at least lend a hand.

Sincerely,

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
United States PostgreSQL Association: http://www.postgresql.us/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate

Bruce Momjian <bruce@momjian.us> writes:

Tom Lane wrote:

No, changing that wasn't what I meant to suggest. My point here is that
we'd dropped a number of big mushy discussions into the queue with the
idea that they'd be re-opened during commit fest, but new discussion
did not happen to any useful extent. Conclusion: don't bother putting
anything less concrete than a patch on the fest queue.

So when/how do those discussions get resolved?

[ shrug... ] You can't force ideas to happen on a schedule.

If you don't want an issue to get forgotten, then make a TODO entry for
it. But the purpose of commit fest is to make sure we deal with things
that can be dealt with in a timely fashion. It's not going to cause
solutions to unsolved problems to appear from nowhere.

regards, tom lane