ideas for auto-processing patches

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

Thanks,

Gavin

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

Another idea is using the git-cvs interoperability system, as described
here (albeit with SVN, but the idea is the same):

http://tw.apinc.org/weblog/2007/01/03#subverting-git

Now, if we were to use a distributed system like Monotone this sort of
thing would be completely a non-issue ...

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Thu, 4 Jan 2007, Alvaro Herrera wrote:

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

Another idea is using the git-cvs interoperability system, as described
here (albeit with SVN, but the idea is the same):

http://tw.apinc.org/weblog/2007/01/03#subverting-git

It seems like that will just add one more cog to the machinary for no
extra benefit. Am I missing something?

Now, if we were to use a distributed system like Monotone this sort of
thing would be completely a non-issue ...

Monotone is so 2006. The new new thing is mercurial!

Gavin

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.

cheers

andrew

On Thu, 4 Jan 2007, Andrew Dunstan wrote:

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.

The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.

It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.

Thanks,

Gavin

On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:

On Thu, 4 Jan 2007, Andrew Dunstan wrote:

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability.

Yeah, that pretty much covers it.

Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.

Right some sort of local mirror would definitely speed things up.

The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.

It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.

PLM's authentication is tied to OSDL's internal authentication system,
but some I imagine setting up accounts and trusting specific users
would be an easy first try.

Regards,
Mark

Gavin Sherry wrote:

With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.

I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.

The patches would need to be vetted first, or no sane buildfarm owner
will
want to use them.

It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.

Beyond committers?

cheers

andrew

On Thu, 4 Jan 2007, Andrew Dunstan wrote:

Gavin Sherry wrote:

With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.

I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.

Right... my comment was more for Mark.

It would be nice if there could be a class of trusted users whose patches
would not have to be vetted.

Beyond committers?

Hmmm... good question. I think so. I imagine the group would be small
though.

Thanks,

Gavin

markwkm@gmail.com schrieb:

On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:

On Thu, 4 Jan 2007, Andrew Dunstan wrote:

...

Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you
say, a
local mirror would be beneficial for patch testing.

Right some sort of local mirror would definitely speed things up.

Easier speedup in this regard would be using subversion instead
of cvs. It transfers only diffs to your working copy (or rather,
to your last checkout) so its really saving on bandwidth.

Regards
Tino

Andrew Dunstan wrote:

Gavin Sherry wrote:

With PLM, you could test patches against various code branches. I'd
guessed Mark would want to provide this capability. Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you say, a
local mirror would be beneficial for patch testing.

I think you're missing the point. Buildfarm members already typically have
or can get very cheaply a copy of each branch they build (HEAD and/or
REL*_*_STABLE). As long as the patch feed is kept to just patches which
they can apply there should be no great bandwidth issues.

yeah - another thing to consider is that switching to a different scm
repository qould put quite a burden on the buildfarm admins (most of
those are not that easily available for the more esotheric platforms for
example).
I'm also not sure how useful it would be to test patches against
branches other then HEAD - new and complex patches will only get applied
on HEAD anyway ...

Stefan

Tino Wildenhain wrote:

markwkm@gmail.com schrieb:

On 1/4/07, Gavin Sherry <swm@linuxworld.com.au> wrote:

On Thu, 4 Jan 2007, Andrew Dunstan wrote:

...

Pulling branches from
anonvcvs regularly might be burdensome bandwidth-wise. So, like you
say, a
local mirror would be beneficial for patch testing.

Right some sort of local mirror would definitely speed things up.

Easier speedup in this regard would be using subversion instead
of cvs. It transfers only diffs to your working copy (or rather,
to your last checkout) so its really saving on bandwidth.

cvs update isn't too bad either. I just did a substantial update on a
tree that had not been touched for nearly 6 months, and ethereal tells
me that total traffic was 7343004 bytes in 7188 packets. Individual
buildfarm updates are going to be much lower than that, by a couple of
orders of magnitude, I suspect.

If we were to switch to subversion we should do it for the right reason
- this isn't one.

cheers

andrew

On Jan 5, 2007, at 10:24 AM, Andrew Dunstan wrote:

cvs update isn't too bad either. I just did a substantial update on
a tree that had not been touched for nearly 6 months, and ethereal
tells me that total traffic was 7343004 bytes in 7188 packets.
Individual buildfarm updates are going to be much lower than that,
by a couple of orders of magnitude, I suspect.

More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it. So if this were added to buildfarm,
presumably it would build plain CVS, then start testing patches. It
could try a CVS up between each patch to see if anything changed, and
possibly start back at the top at that point.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

Jim Nasby wrote:

On Jan 5, 2007, at 10:24 AM, Andrew Dunstan wrote:

cvs update isn't too bad either. I just did a substantial update on
a tree that had not been touched for nearly 6 months, and ethereal
tells me that total traffic was 7343004 bytes in 7188 packets.
Individual buildfarm updates are going to be much lower than that,
by a couple of orders of magnitude, I suspect.

More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it. So if this were added to buildfarm,
presumably it would build plain CVS, then start testing patches. It
could try a CVS up between each patch to see if anything changed, and
possibly start back at the top at that point.

Actually, I think a patch would need to be designated against a particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.

Certainly patch processing would be both optional and something done
separately from standard CVS branch processing.

cheers

andrew

"Andrew Dunstan" <andrew@dunslane.net> writes:

Jim Nasby wrote:

More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.

Actually, I think a patch would need to be designated against a particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.

I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.

There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.

regards, tom lane

Tom Lane wrote:

"Andrew Dunstan" <andrew@dunslane.net> writes:

Jim Nasby wrote:

More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.

Actually, I think a patch would need to be designated against a
particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.

I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.

There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.

OK, if the aim is to catch patch bitrot, then you're right, of course.

cheers

andrew

On Fri, Jan 05, 2007 at 11:02:32PM -0600, Andrew Dunstan wrote:

Tom Lane wrote:

"Andrew Dunstan" <andrew@dunslane.net> writes:

Jim Nasby wrote:

More important, I see no reason to tie applying patches to pulling
from CVS. In fact, I think it's a bad idea: you want to build just
what's in CVS first, to make sure that it's working, before you start
testing any patches against it.

Actually, I think a patch would need to be designated against a
particular
branch and timestamp, and the buildfarm member would need to "update" to
that on its temp copy before applying the patch.

I think I like Jim's idea better: you want to find out if some other
applied patch has broken the patch-under-test, so I cannot see a reason
for testing against anything except branch tip.

There certainly is value in being able to test against a non-HEAD branch
tip, but I don't see the point in testing against a back timestamp.

OK, if the aim is to catch patch bitrot, then you're right, of course.

Actually, I see point in both... I'd think you'd want to know if a patch
worked against the CVS checkout it was written against. But of course
each member would only need to test that once. You'd also want to set
something up to capture the exact timestamp that a repo was checked out
at so that you could submit that info along with your patch (btw, a plus
to subversion is that you'd be able to refer to the exact checkout with
a single version number).

But since setting that up would require non-trivial additional work, I'd
just save it for latter and get testing against the latest HEAD up and
running.
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:

Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.

Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed. (And isn't the version number
included in the patch if generated as a diff anyway?)

Michael Glaesemann
grzm seespotcode net

On Mon, Jan 08, 2007 at 10:40:16PM -0600, Michael Glaesemann wrote:

On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:

Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.

Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed.

Wouldn't there be some value to knowing whether the patch failed due to
bitrot vs it just didn't work on some platforms out of the gate?

(And isn't the version number
included in the patch if generated as a diff anyway?)

Of the patched files, yes... but that says little if anything about the
rest of the tree... unless the patch includes a file that is forced to
change every time there's a commit... but then the patch creator would
also have to change that file, which would create a mess. Yuck.

This is why associating a patch with a specific version of the tree
should definitely wait for version 2 of the patchfarm (or should it be
called the farmers patch? ;) ).
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Jan 9, 2007, at 20:41 , Jim C. Nasby wrote:

On Mon, Jan 08, 2007 at 10:40:16PM -0600, Michael Glaesemann wrote:

On Jan 8, 2007, at 19:25 , Jim C. Nasby wrote:

Actually, I see point in both... I'd think you'd want to know if a
patch
worked against the CVS checkout it was written against.

Regardless, it's unlikely that the patch was tested against all of
the platforms available on the build farm. If it fails on some of the
build|patch farm animals, or if it fails due to bitrot, the point is
it fails: whatever version the patch was generated against is pretty
much moot: the patch needs to be fixed.

Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?

I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?

Michael Glaesemann
grzm seespotcode net

On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote:

Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?

I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?

I was thinking that knowing it did work at one time would be useful, but
maybe that's not the case...
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

On Wed, 10 Jan 2007, Jim C. Nasby wrote:

On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote:

Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?

I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?

I was thinking that knowing it did work at one time would be useful, but
maybe that's not the case...

It might be useful to patch authors who submit code which remains
unreviewed for some time. Then the submitter or reviewer will be able to
know at what date the code drifted. This might be easier than looking
through the commit history and trying to locate the problem, I think.

Still, the more interesting thing to me would be to be able to provide in
the patch a set of custom tests inside of the regression test frame work
which aren't suitable as RTs in the long term but will be able to tell the
patch author if their code works correctly on a variety of platforms.

Thanks,

Gavin

On Wed, 10 Jan 2007, Jim C. Nasby wrote:

On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote:

Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?

I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?

I was thinking that knowing it did work at one time would be useful, but
maybe that's not the case...

"Has it ever worked" is the singularly most fundamental technical support
question; yes, it has value.

One question here - rhetorical, perhaps - is; What changed and when? Often
when things changed can help get you to what changed. (This is what logs
are for, and not just automated computer logs, but system management
things like, "I upgraded GCC today.") And that can help you focus in on
what to do to fix the problem. (such as looking to the GCC release notes)

A non-rhetorical question is; Shouldn't the build process mechanism/system
know when _any_ aspect of a build has failed (including patches)? I'd
think so, especially in a build-farm scenario.

...Just my two cents - and worth every penny! -smile-

Richard

--
Richard Troy, Chief Scientist
Science Tools Corporation
510-924-1363 or 202-747-1263
rtroy@ScienceTools.com, http://ScienceTools.com/

On Jan 11, 2007, at 10:35 , Richard Troy wrote:

On Wed, 10 Jan 2007, Jim C. Nasby wrote:

On Thu, Jan 11, 2007 at 08:04:41AM +0900, Michael Glaesemann wrote:

Wouldn't there be some value to knowing whether the patch failed
due to
bitrot vs it just didn't work on some platforms out of the gate?

I'm having a hard time figuring out what that value would be. How
would that knowledge affect what's needed to fix the patch?

I was thinking that knowing it did work at one time would be
useful, but
maybe that's not the case...

"Has it ever worked" is the singularly most fundamental technical
support
question; yes, it has value.

You'd be able to see whether or not it ever worked by when the patch
first hit the patch farm.

One question here - rhetorical, perhaps - is; What changed and when?

This is recorded in the current build farm.

Michael Glaesemann
grzm seespotcode net

On 1/4/07, Andrew Dunstan <andrew@dunslane.net> wrote:

Gavin Sherry wrote:

On Thu, 4 Jan 2007 markwkm@gmail.com wrote:

1. Pull source directly from repositories (cvs, git, etc.) PLM
doesn't really track actually scm repositories. It requires
directories of source code to be traversed, which are set up by
creating mirrors.

It seems to me that a better approach might be to mirror the CVS repo --
or at least make that an option -- and pull the sources locally. Having to
pull down >100MB of data for every build might be onerous to some build
farm members.

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

I'm thinking that a SOAP call might be easier to implement? The RSS
feed seems like it would be more interesting as I am imagining that a
buildfarm system might be able to react to new patches being added to
the system. But maybe that's a trivial thing for either SOAP or an
RSS feed.

The patches would need to be vetted first, or no sane buildfarm owner will
want to use them.

Perhaps as a first go it can pull any patch that can be applied
without errors? The list of patches to test can be eventually
restricted by name and who submitted them.

Regards,
Mark

markwkm@gmail.com wrote:

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or
rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

I'm thinking that a SOAP call might be easier to implement? The RSS
feed seems like it would be more interesting as I am imagining that a
buildfarm system might be able to react to new patches being added to
the system. But maybe that's a trivial thing for either SOAP or an
RSS feed.

I'd be quite happy with SOAP. We can make SOAP::Lite an optional load
module, so if you don't want to run patches you don't need to have the
module available.

The patches would need to be vetted first, or no sane buildfarm owner
will
want to use them.

Perhaps as a first go it can pull any patch that can be applied
without errors? The list of patches to test can be eventually
restricted by name and who submitted them.

This reasoning seems unsafe. I am not prepared to test arbitrary patches
on my machine - that seems like a perfect recipe for a trojan horse. I
want to know that they have been vetted by someone I trust. That means
that in order to get into the feed in the first place there has to be a
group of trusted submitters. Obviously, current postgres core committers
should be in that group, and I can think of maybe 5 or 6 other people
that could easily be on it. Perhaps we should leave the selection to the
core team.

cheers

andrew

On 1/11/07, Andrew Dunstan <andrew@dunslane.net> wrote:

markwkm@gmail.com wrote:

I am not clear about what is being proposed. Currently buildfarm syncs
against (or pulls a fresh copy from, depending on configuration) either
the main anoncvs repo or a mirror (which you can get using cvsup or
rsync,
among other mechanisms). I can imagine a mechanism in which we pull
certain patches from a patch server (maybe using an RSS feed, or a SOAP
call?) which could be applied before the run. I wouldn't want to couple
things much more closely than that.

I'm thinking that a SOAP call might be easier to implement? The RSS
feed seems like it would be more interesting as I am imagining that a
buildfarm system might be able to react to new patches being added to
the system. But maybe that's a trivial thing for either SOAP or an
RSS feed.

I'd be quite happy with SOAP. We can make SOAP::Lite an optional load
module, so if you don't want to run patches you don't need to have the
module available.

The patches would need to be vetted first, or no sane buildfarm owner
will
want to use them.

Perhaps as a first go it can pull any patch that can be applied
without errors? The list of patches to test can be eventually
restricted by name and who submitted them.

This reasoning seems unsafe. I am not prepared to test arbitrary patches
on my machine - that seems like a perfect recipe for a trojan horse. I
want to know that they have been vetted by someone I trust. That means
that in order to get into the feed in the first place there has to be a
group of trusted submitters. Obviously, current postgres core committers
should be in that group, and I can think of maybe 5 or 6 other people
that could easily be on it. Perhaps we should leave the selection to the
core team.

That's an excellent point; I didn't think of the trojan horse
scenario. What do you think about setting up the buildfarm clients
with the users they are willing to test patches for, as opposed to
having the patch system track who is are trusted users? My thoughts
are the former is easier to implement and that it allows anyone to use
the buildfarm to test a patch for anyone, well each buildfarm client
user permitting.

Regards,
Mark

markwkm@gmail.com wrote:

What do you think about setting up the buildfarm clients
with the users they are willing to test patches for, as opposed to
having the patch system track who is are trusted users? My thoughts
are the former is easier to implement and that it allows anyone to use
the buildfarm to test a patch for anyone, well each buildfarm client
user permitting.

We can do this, but the utility will be somewhat limited. The submitters
will still have to be known and authenticated on the patch server. I
think you're also overlooking one of the virtues of the buildfarm,
namely that it does its thing unattended. If there is a preconfigured
set of submitters/vetters then we can rely on them all to do their
stuff. If it's more ad hoc, then when Joe Bloggs submits a spiffy new
patch every buildfarm owner that wanted to test it would need to go and
add him to their configured list of patch submitters. This doesn't seem
too workable.

cheers

andrew

On 1/12/07, Andrew Dunstan <andrew@dunslane.net> wrote:

markwkm@gmail.com wrote:

What do you think about setting up the buildfarm clients
with the users they are willing to test patches for, as opposed to
having the patch system track who is are trusted users? My thoughts
are the former is easier to implement and that it allows anyone to use
the buildfarm to test a patch for anyone, well each buildfarm client
user permitting.

We can do this, but the utility will be somewhat limited. The submitters
will still have to be known and authenticated on the patch server. I
think you're also overlooking one of the virtues of the buildfarm,
namely that it does its thing unattended. If there is a preconfigured
set of submitters/vetters then we can rely on them all to do their
stuff. If it's more ad hoc, then when Joe Bloggs submits a spiffy new
patch every buildfarm owner that wanted to test it would need to go and
add him to their configured list of patch submitters. This doesn't seem
too workable.

Ok so it really wasn't much work to put together a SOAP call that'll
return patches from everyone, a trusted group, or a specified
individual. I put together a short perl example that illustrates some
of this:
http://folio.dyndns.org/example.pl.txt

How does that look?

Regards,
Mark

markwkm@gmail.com wrote:

On 1/12/07, Andrew Dunstan <andrew@dunslane.net> wrote:

markwkm@gmail.com wrote:

What do you think about setting up the buildfarm clients
with the users they are willing to test patches for, as opposed to
having the patch system track who is are trusted users? My thoughts
are the former is easier to implement and that it allows anyone to use
the buildfarm to test a patch for anyone, well each buildfarm client
user permitting.

We can do this, but the utility will be somewhat limited. The submitters
will still have to be known and authenticated on the patch server. I
think you're also overlooking one of the virtues of the buildfarm,
namely that it does its thing unattended. If there is a preconfigured
set of submitters/vetters then we can rely on them all to do their
stuff. If it's more ad hoc, then when Joe Bloggs submits a spiffy new
patch every buildfarm owner that wanted to test it would need to go and
add him to their configured list of patch submitters. This doesn't seem
too workable.

Ok so it really wasn't much work to put together a SOAP call that'll
return patches from everyone, a trusted group, or a specified
individual. I put together a short perl example that illustrates some
of this:
http://folio.dyndns.org/example.pl.txt

How does that look?

Looks OK in general, although I would need to know a little more of the
semantics. I get back a structure that looks like what's below.

One thing: the patch server will have to run over HTTPS - that way we
can know that it is who it says it is.

cheers

andrew

$VAR1 = [
bless( {
'repository_id' => '1',
'created_on' => '2007-01-15T19:40:09-08:00',
'diff' => 'dummied out',
'name' => 'copy_nowal.v1.patch',
'owner_id' => '1',
'id' => '1',
'updated_on' => '2007-01-15T11:40:10-08:00'
}, 'Patch' ),
bless( {
'repository_id' => '1',
'created_on' => '2007-01-15T19:40:09-08:00',
'diff' => 'dummied out',
'name' => 'pgsql-bitmap-09-17.patch',
'owner_id' => '1',
'id' => '2',
'updated_on' => '2007-01-15T11:40:29-08:00'
}, 'Patch' )
];

On 1/17/07, Andrew Dunstan <andrew@dunslane.net> wrote:

markwkm@gmail.com wrote:

On 1/12/07, Andrew Dunstan <andrew@dunslane.net> wrote:

markwkm@gmail.com wrote:

What do you think about setting up the buildfarm clients
with the users they are willing to test patches for, as opposed to
having the patch system track who is are trusted users? My thoughts
are the former is easier to implement and that it allows anyone to use
the buildfarm to test a patch for anyone, well each buildfarm client
user permitting.

We can do this, but the utility will be somewhat limited. The submitters
will still have to be known and authenticated on the patch server. I
think you're also overlooking one of the virtues of the buildfarm,
namely that it does its thing unattended. If there is a preconfigured
set of submitters/vetters then we can rely on them all to do their
stuff. If it's more ad hoc, then when Joe Bloggs submits a spiffy new
patch every buildfarm owner that wanted to test it would need to go and
add him to their configured list of patch submitters. This doesn't seem
too workable.

Ok so it really wasn't much work to put together a SOAP call that'll
return patches from everyone, a trusted group, or a specified
individual. I put together a short perl example that illustrates some
of this:
http://folio.dyndns.org/example.pl.txt

How does that look?

Looks OK in general, although I would need to know a little more of the
semantics. I get back a structure that looks like what's below.

There probably isn't a need to return all that data. I was being lazy
and returning the entire object. I'll annotate below.

One thing: the patch server will have to run over HTTPS - that way we
can know that it is who it says it is.

Right, I'm not sure if the computer I'm proofing it on is the best
place for it so I didn't bother with the HTTPS, but should be trivial
to have it.

cheers

andrew

$VAR1 = [
bless( {
'repository_id' => '1',

ID of the repository the patch applies to.

'created_on' => '2007-01-15T19:40:09-08:00',

Timestamp of when the record was created.

'diff' => 'dummied out',

Actual patch, in plain text.

'name' => 'copy_nowal.v1.patch',

Name of the patch file.

'owner_id' => '1',

ID of the owner of the patch.

'id' => '1',

ID of the patch.

'updated_on' => '2007-01-15T11:40:10-08:00'

Timestamp of when patch was updated.

}, 'Patch' ),
bless( {
'repository_id' => '1',
'created_on' => '2007-01-15T19:40:09-08:00',
'diff' => 'dummied out',
'name' => 'pgsql-bitmap-09-17.patch',
'owner_id' => '1',
'id' => '2',
'updated_on' => '2007-01-15T11:40:29-08:00'
}, 'Patch' )
];

Regards,
Mark

markwkm@gmail.com wrote:

One thing: the patch server will have to run over HTTPS - that way we
can know that it is who it says it is.

Right, I'm not sure if the computer I'm proofing it on is the best
place for it so I didn't bother with the HTTPS, but should be trivial
to have it.

Yes, this was more by way of a "don't forget this" note. The
implementation can be happily oblivious of it, other than setting https
in the proxy for the SOAP::Lite dispatcher. From a buildfarm point of
view, we would add such SOAP params into the config file.

cheers

andrew