ideas for auto-processing patches
OSDL had a tool called PLM with a primary goal to test patches against
the Linux kernel. It applied them and built them on multiple
platforms. It's a pretty simple idea and here are some links to what
it did; the systems appear to still be up for the moment so here are a
couple of links to what it did.
Summary of build results:
http://plm.testing.osdl.org/patches/show/linux-2.6.20-rc3-git3
Summary of recent patches submitted into the system:
http://plm.testing.osdl.org/patches/search_result
It also provides an rss feed:
http://plm.testing.osdl.org/rss
There a couple of things initial things I wanted to change, which I
think are improvements:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.
2. Apply and build patches against daily updates from the
repositories, as opposed to only against a specified version of the
source code.
Thoughts?
Regards,
Mark
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.
It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.
Thanks,
Gavin
Gavin Sherry wrote:
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.
Another idea is using the git-cvs interoperability system, as described
here (albeit with SVN, but the idea is the same):
http://tw.apinc.org/weblog/2007/01/03#subverting-git
Now, if we were to use a distributed system like Monotone this sort of
thing would be completely a non-issue ...
--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
On Thu, 4 Jan 2007, Alvaro Herrera wrote:
Gavin Sherry wrote:
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.Another idea is using the git-cvs interoperability system, as described
here (albeit with SVN, but the idea is the same):
It seems like that will just add one more cog to the machinary for no
extra benefit. Am I missing something?
Now, if we were to use a distributed system like Monotone this sort of
thing would be completely a non-issue ...
Monotone is so 2006. The new new thing is mercurial!
Gavin
Gavin Sherry wrote:
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.
I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.
The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.
cheers
andrew
On Thu, 4 Jan 2007, Andrew Dunstan wrote:
Gavin Sherry wrote:
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.
With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.
The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.
It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.
Thanks,
Gavin
On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:
On Thu, 4 Jan 2007, Andrew Dunstan wrote:
Gavin Sherry wrote:
On Thu, 4 Jan 2007 markwkm@gmail.com wrote:
1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability.
Yeah, that pretty much covers it.
Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.
Right some sort of local mirror would definitely speed things up.
The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.
PLM's authentication is tied to OSDL's internal authentication system,
but some I imagine setting up accounts and trusting specific users
would be an easy first try.
Regards,
Mark
Gavin Sherry wrote:
With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.
I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.
The patches would need to be vetted first, or no sane buildfarm owner
will
want to use them.It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.
Beyond committers?
cheers
andrew
On Thu, 4 Jan 2007, Andrew Dunstan wrote:
Gavin Sherry wrote:
With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.
Right... my comment was more for Mark.
It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.Beyond committers?
Hmmm... good question. I think so. I imagine the group would be small
though.
Thanks,
Gavin
markwkm@gmail.com schrieb:
On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:
On Thu, 4 Jan 2007, Andrew Dunstan wrote:
...
Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you
say, a
local mirror would be beneficial for patch testing.Right some sort of local mirror would definitely speed things up.
Easier speedup in this regard would be using subversion instead
of cvs. It transfers only diffs to your working copy (or rather,
to your last checkout) so its really saving on bandwidth.
Regards
Tino
Andrew Dunstan wrote:
Gavin Sherry wrote:
With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.
yeah - another thing to consider is that switching to a different scm
repository qould put quite a burden on the buildfarm admins (most of
those are not that easily available for the more esotheric platforms for
example).
I'm also not sure how useful it would be to test patches against
branches other then HEAD - new and complex patches will only get applied
on HEAD anyway ...
Stefan
Tino Wildenhain wrote:
markwkm@gmail.com schrieb:
On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:
On Thu, 4 Jan 2007, Andrew Dunstan wrote:
...
Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you
say, a
local mirror would be beneficial for patch testing.Right some sort of local mirror would definitely speed things up.
Easier speedup in this regard would be using subversion instead
of cvs. It transfers only diffs to your working copy (or rather,
to your last checkout) so its really saving on bandwidth.
cvs update isn't too bad either. I just did a substantial update on a
tree that had not been touched for nearly 6 months, and ethereal tells
me that total traffic was 7343004 bytes in 7188 packets. Individual
buildfarm updates are going to be much lower than that, by a couple of
orders of magnitude, I suspect.
If we were to switch to subversion we should do it for the right reason
- this isn't one.
cheers
andrew
On Jan 5, 2007, at 10:24 AM, Andrew Dunstan wrote:
cvs update isn't too bad either. I just did a substantial update on
a tree that had not been touched for nearly 6 months, and ethereal
tells me that total traffic was 7343004 bytes in 7188 packets.
Individual buildfarm updates are going to be much lower than that,
by a couple of orders of magnitude, I suspect.
More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it. So if this were added to buildfarm,
presumably it would build plain CVS, then start testing patches. It
could try a CVS up between each patch to see if anything changed, and
possibly start back at the top at that point.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
Jim Nasby wrote:
On Jan 5, 2007, at 10:24 AM, Andrew Dunstan wrote:
cvs update isn't too bad either. I just did a substantial update on
a tree that had not been touched for nearly 6 months, and ethereal
tells me that total traffic was 7343004 bytes in 7188 packets.
Individual buildfarm updates are going to be much lower than that,
by a couple of orders of magnitude, I suspect.More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it. So if this were added to buildfarm,
presumably it would build plain CVS, then start testing patches. It
could try a CVS up between each patch to see if anything changed, and
possibly start back at the top at that point.
Actually, I think a patch would need to be designated against a particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.
Certainly patch processing would be both optional and something done
separately from standard CVS branch processing.
cheers
andrew
Import Notes
Resolved by subject fallback
"Andrew Dunstan" <andrew@dunslane.net> writes:
Jim Nasby wrote:
More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.
Actually, I think a patch would need to be designated against a particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.
I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.
There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.
regards, tom lane
Tom Lane wrote:
"Andrew Dunstan" <andrew@dunslane.net> writes:
Jim Nasby wrote:
More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.Actually, I think a patch would need to be designated against a
particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.
OK, if the aim is to catch patch bitrot, then you're right, of course.
cheers
andrew
On Fri, Jan 05, 2007 at 11:02:32PM -0600, Andrew Dunstan wrote:
Tom Lane wrote:
"Andrew Dunstan" <andrew@dunslane.net> writes:
Jim Nasby wrote:
More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.Actually, I think a patch would need to be designated against a
particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.OK, if the aim is to catch patch bitrot, then you're right, of course.
Actually, I see point in both... I'd think you'd want to know if a patch
worked against the CVS checkout it was written against. But of course
each member would only need to test that once. You'd also want to set
something up to capture the exact timestamp that a repo was checked out
at so that you could submit that info along with your patch (btw, a plus
to subversion is that you'd be able to refer to the exact checkout with
a single version number).
But since setting that up would require non-trivial additional work, I'd
just save it for latter and get testing against the latest HEAD up and
running.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:
Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.
Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed. (And isn't the version number
included in the patch if generated as a diff anyway?)
Michael Glaesemann
grzm seespotcode net
On Mon, Jan 08, 2007 at 10:40:16PM -0600, Michael Glaesemann wrote:
On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:
Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed.
Wouldn't there be some value to knowing whether the patch failed due to
bitrot vs it just didn't work on some platforms out of the gate?
(And isn't the version number
included in the patch if generated as a diff anyway?)
Of the patched files, yes... but that says little if anything about the
rest of the tree... unless the patch includes a file that is forced to
change every time there's a commit... but then the patch creator would
also have to change that file, which would create a mess. Yuck.
This is why associating a patch with a specific version of the tree
should definitely wait for version 2 of the patchfarm (or should it be
called the farmers patch? ;) ).
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
On Jan 9, 2007, at 20:41 , Jim C. Nasby wrote:
On Mon, Jan 08, 2007 at 10:40:16PM -0600, Michael Glaesemann wrote:
On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:
Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed.Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?
I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?
Michael Glaesemann
grzm seespotcode net
