Git cvsserver serious issue
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.
Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?
As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.
Ugh.
Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?
As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?
If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.
Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.
regards, tom lane
On Wed, Sep 22, 2010 at 16:23, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.Ugh.
Indeed.
Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.
We could. It's currently on a freebsd vm though and I don't think we
can set per-server IP filters on those. (I was thinking iptables). We
could move it though - it doesn't *have* to be on the anonymous git
VM. It's just some extra resources.
Well, the use-case I was thinking of was Stefan. While he can't run
git on each and every animal, he certainly has *some* machine(s) on
the correct side of whatever firewall there may be that can run git.
Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.
We could do that, but that could end up fighting a losing battle in
case some bot hits it.
I don't like deploying something with a known issue on it, sandboxed or not.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 09/22/2010 10:23 AM, Tom Lane wrote:
If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.
I'm not convinced we need any such thing yet. 13 of the 38 animals that
have reported in the last 5 days are using git already (OK, factoring
out my animals that's 8 out of 33). I'm going to send out email in a few
days prodding people to migrate. Let's see how far we get.
cheers
andrew
Andrew Dunstan <andrew@dunslane.net> writes:
On 09/22/2010 10:23 AM, Tom Lane wrote:
If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.
I'm not convinced we need any such thing yet. 13 of the 38 animals that
have reported in the last 5 days are using git already (OK, factoring
out my animals that's 8 out of 33). I'm going to send out email in a few
days prodding people to migrate. Let's see how far we get.
Even if we get 100% compliance on the buildfarm side, Peter's already
stated that moving the NLS support over to git is going to take more
time than we have available right now. We need a cvsserver for awhile
yet. We can't just suddenly announce "CVS service is terminated as of
yesterday" and expect that that's not going to have any serious
consequences.
Is there anything we could do to patch the problem out of git-cvsserver?
Maybe hack it to only accept requests for the active branch tips?
regards, tom lane
On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?
Well, let's see. If someone can figure out the git equivalent of
if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi
(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)
then I might be able to get this fixed.
On Wed, Sep 22, 2010 at 12:21 PM, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?Well, let's see. If someone can figure out the git equivalent of
if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)then I might be able to get this fixed.
Can you just check whether the commit SHA of HEAD has changed? e.g.
git show-ref --heads -s master
git log --format=format:%H -n 1 master
...and compare with previous results of same?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company
Excerpts from Peter Eisentraut's message of mié sep 22 12:21:45 -0400 2010:
Well, let's see. If someone can figure out the git equivalent of
if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)then I might be able to get this fixed.
Would it work to save the previous commit hash in a file and compare to
the current one?
--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
At 2010-09-22 19:21:45 +0300, peter_e@gmx.net wrote:
Well, let's see. If someone can figure out the git equivalent of
if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi
I think you want:
git pull
if [ $(git rev-parse HEAD) != $(git rev-parse ORIG_HEAD) ]; then
# ... the pull changed something ...
fi
-- ams
On 09/22/2010 10:26 AM, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 16:23, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.Ugh.
Indeed.
Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?
As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.We could. It's currently on a freebsd vm though and I don't think we
can set per-server IP filters on those. (I was thinking iptables). We
could move it though - it doesn't *have* to be on the anonymous git
VM. It's just some extra resources.Well, the use-case I was thinking of was Stefan. While he can't run
git on each and every animal, he certainly has *some* machine(s) on
the correct side of whatever firewall there may be that can run git.Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.We could do that, but that could end up fighting a losing battle in
case some bot hits it.I don't like deploying something with a known issue on it, sandboxed or not.
Thinking about this some more, how about we do non-anonymous CVS over
SSH access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?
I'm not sure if that would handle other requirements, such as Peter's,
but I hope the residual requirements for CVS support will be pretty rare.
cheers
andrew
On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan <andrew@dunslane.net> wrote:
Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.We could do that, but that could end up fighting a losing battle in
case some bot hits it.I don't like deploying something with a known issue on it, sandboxed or
not.Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?
Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).
I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...
I'm not sure if that would handle other requirements, such as Peter's, but I
hope the residual requirements for CVS support will be pretty rare.
Just to be sure - do we have any other requirements for CVS *beyond*
buildfarm and NLS that we're not thinking of here?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 09/23/2010 02:09 AM, Magnus Hagander wrote:
On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net> wrote:
Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.We could do that, but that could end up fighting a losing battle in
case some bot hits it.I don't like deploying something with a known issue on it, sandboxed or
not.Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...
Yes, I think we should have a VM. Is that so hard to do in these days of
Xen etc? I'm surprised we can't run up a VM pretty much at the drop of a
hat.
I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can
be sniffed or attacked using brute force. IP addresses can be spoofed.
But you're the SA, not me.
cheers
andrew
On Thu, Sep 23, 2010 at 11:27, Andrew Dunstan <andrew@dunslane.net> wrote:
On 09/23/2010 02:09 AM, Magnus Hagander wrote:
On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net> wrote:
Also, couldn't we just set up the cvsserver on its own VM with a
limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.We could do that, but that could end up fighting a losing battle in
case some bot hits it.I don't like deploying something with a known issue on it, sandboxed or
not.Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...Yes, I think we should have a VM. Is that so hard to do in these days of Xen
etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.
In the new infrastructure, it is.
The main resource that's limited really is IP addresses ;)
I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can be
sniffed or attacked using brute force. IP addresses can be spoofed. But
you're the SA, not me.
I prefer keys. But I don't want those users on the same VM as
important services, because passphrase-less keys are a lot more likely
to be compromised than the keys of say a regular committer...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander wrote:
I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...Yes, I think we should have a VM. Is that so hard to do in these days of Xen
etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.In the new infrastructure, it is.
The main resource that's limited really is IP addresses ;)
I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can be
sniffed or attacked using brute force. IP addresses can be spoofed. But
you're the SA, not me.I prefer keys. But I don't want those users on the same VM as
important services, because passphrase-less keys are a lot more likely
to be compromised than the keys of say a regular committer...
Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On 09/23/2010 09:55 AM, Bruce Momjian wrote:
Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?
Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.
cheers
andrew
Andrew Dunstan wrote:
On 09/23/2010 09:55 AM, Bruce Momjian wrote:
Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.
I suppose there is no way to pull diffs from git continuously and commit
them to CVS.
--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:
Andrew Dunstan wrote:
On 09/23/2010 09:55 AM, Bruce Momjian wrote:
Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.I suppose there is no way to pull diffs from git continuously and commit
them to CVS.
You probably could. I'm very doubtful it's worth it...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Thu, Sep 23, 2010 at 04:38:27PM +0200, Magnus Hagander wrote:
On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:
Andrew Dunstan wrote:
On 09/23/2010 09:55 AM, Bruce Momjian wrote:
Stupid question, but can't we just create a CVSROOT fed from
git, and use the normal CVS server to feed sites?Where is it going to get the ,v files that CVS uses from?
git-cvsserver emulates a CVS server from git. It doesn't create a
CVS repository that CVS in server mode could use.I suppose there is no way to pull diffs from git continuously and
commit them to CVS.You probably could. I'm very doubtful it's worth it...
Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?
Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
Magnus Hagander <magnus@hagander.net> writes:
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.
I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script. I could probably hack it in an hour,
there are those here who could do it in ten minutes.
regards, tom lane
On 09/23/2010 10:58 AM, David Fetter wrote:
Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?
At least one buildfarm member, spoonbill, is known to have issues with
git. This machine, because of the flags it uses, has found numerous bugs
for us in the past and is quite important for us to maintain (a direct
counter argument to your suggestion of dropping platforms that don't
support git).
cheers
andrew
On Thu, Sep 23, 2010 at 17:16, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script. I could probably hack it in an hour,
there are those here who could do it in ten minutes.
Yeah, that would not be a bad idea - if someone can do it who feels
comfortable doing it :-)
I could probably hack it up as well, but I wouldn't trust myself to
have convered all the bases.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Thu, Sep 23, 2010 at 11:17:35AM -0400, Andrew Dunstan wrote:
On 09/23/2010 10:58 AM, David Fetter wrote:
Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?At least one buildfarm member, spoonbill, is known to have issues
with git.
Do those issues appear fixable?
Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
On 09/23/2010 11:18 AM, Magnus Hagander wrote:
On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script. I could probably hack it in an hour,
there are those here who could do it in ten minutes.Yeah, that would not be a bad idea - if someone can do it who feels
comfortable doing it :-)I could probably hack it up as well, but I wouldn't trust myself to
have convered all the bases.
Are we sure that's going to stop the DOS issue?
cheers
andrew
David Fetter wrote:
On Thu, Sep 23, 2010 at 11:17:35AM -0400, Andrew Dunstan wrote:
On 09/23/2010 10:58 AM, David Fetter wrote:
Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?At least one buildfarm member, spoonbill, is known to have issues
with git.Do those issues appear fixable?
maybe but I have absolutely no time to investigate in the next few days.
Stefan
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
On 09/23/2010 11:18 AM, Magnus Hagander wrote:
On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.
Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script. I could probably hack it in an hour,
there are those here who could do it in ten minutes.Yeah, that would not be a bad idea - if someone can do it who feels
comfortable doing it :-)I could probably hack it up as well, but I wouldn't trust myself to
have convered all the bases.Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Andrew Dunstan <andrew@dunslane.net> writes:
On Thu, Sep 23, 2010 at 17:16, Tom Lane<tgl@sss.pgh.pa.us> wrote:
I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
Are we sure that's going to stop the DOS issue?
The claimed denial of service is that each checkout target requires a
separate SQLite database. Limit the number of checkout targets accepted
and you're done. Or at least, if you're not done, it behooves those
claiming there's a security problem to show what the problem is. It's
not like this piece of software isn't used in production, so I doubt
it needs to be babied quite as much as this thread is assuming.
regards, tom lane
Magnus Hagander <magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.
regards, tom lane
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan <andrew@dunslane.net> wrote:
Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.
Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:
[gitcvs]
enabled=1
modules=master,REL9_0_STABLE,REL8_4_STABLE
--- git-cvsserver.orig 2010-09-23 12:03:06.000000000 -0400
+++ git-cvsserver 2010-09-23 13:16:53.000000000 -0400
@@ -2771,6 +2771,12 @@ die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );
+ if (defined $cfg->{gitcvs}{modules})
+ {
+ $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules});
+ die "Invalid module $module" unless map {/^ *$module$/}
split(',', $cfg->{gitcvs}{modules});
+ }
+
$self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} ||
$cfg->{gitcvs}{dbdriver} || "SQLite";
$self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net> wrote:
Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:
[gitcvs]
enabled=1
modules=master,REL9_0_STABLE,REL8_4_STABLE--- git-cvsserver.orig 2010-09-23 12:03:06.000000000 -0400 +++ git-cvsserver 2010-09-23 13:16:53.000000000 -0400 @@ -2771,6 +2771,12 @@die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );+ if (defined $cfg->{gitcvs}{modules}) + { + $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules}); + die "Invalid module $module" unless map {/^ *$module$/} split(',', $cfg->{gitcvs}{modules}); + } + $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} || $cfg->{gitcvs}{dbdriver} || "SQLite"; $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||
OK, let's go with that. I was kinda hoping that we wouldn't have to do
this at all, but Stefan has been having serious problems getting git to
build and run on spoonbill, and I don't want to take up more of his time
or be without it for very long.
cheers
andrew
On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan <andrew@dunslane.net> wrote:
On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
wrote:Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:
[gitcvs]
enabled=1
modules=master,REL9_0_STABLE,REL8_4_STABLE--- git-cvsserver.orig 2010-09-23 12:03:06.000000000 -0400 +++ git-cvsserver 2010-09-23 13:16:53.000000000 -0400 @@ -2771,6 +2771,12 @@die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );+ if (defined $cfg->{gitcvs}{modules}) + { + $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules}); + die "Invalid module $module" unless map {/^ *$module$/} split(',', $cfg->{gitcvs}{modules}); + } + $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} || $cfg->{gitcvs}{dbdriver} || "SQLite"; $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||OK, let's go with that. I was kinda hoping that we wouldn't have to do this
at all, but Stefan has been having serious problems getting git to build and
run on spoonbill, and I don't want to take up more of his time or be without
it for very long.
Are we doing this *just* for spoonbill? If so, it's a lot easier to
just filter-by-IP, so we don't have to maintain a patched version...
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 10/07/2010 09:44 AM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan<andrew@dunslane.net> wrote:
On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
wrote:Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:
[gitcvs]
enabled=1
modules=master,REL9_0_STABLE,REL8_4_STABLE--- git-cvsserver.orig 2010-09-23 12:03:06.000000000 -0400 +++ git-cvsserver 2010-09-23 13:16:53.000000000 -0400 @@ -2771,6 +2771,12 @@die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );+ if (defined $cfg->{gitcvs}{modules}) + { + $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules}); + die "Invalid module $module" unless map {/^ *$module$/} split(',', $cfg->{gitcvs}{modules}); + } + $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} || $cfg->{gitcvs}{dbdriver} || "SQLite"; $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||OK, let's go with that. I was kinda hoping that we wouldn't have to do this
at all, but Stefan has been having serious problems getting git to build and
run on spoonbill, and I don't want to take up more of his time or be without
it for very long.Are we doing this *just* for spoonbill? If so, it's a lot easier to
just filter-by-IP, so we don't have to maintain a patched version...
That's the only one I know of - there could certainly be others - but if
we're going to support continued CVS use I want to be able to test it,
and I don't have a static IP address. Can you filter by name lookup? I
have a dyndns name.
OTOH, this patch seems pretty small and simple to maintain.
cheers
andrew
On Thu, Oct 7, 2010 at 16:07, Andrew Dunstan <andrew@dunslane.net> wrote:
On 10/07/2010 09:44 AM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 15:16, Andrew Dunstan<andrew@dunslane.net> wrote:
On 09/23/2010 01:18 PM, Aidan Van Dyk wrote:
On Thu, Sep 23, 2010 at 11:49 AM, Tom Lane<tgl@sss.pgh.pa.us> wrote:
Magnus Hagander<magnus@hagander.net> writes:
On Thu, Sep 23, 2010 at 17:32, Andrew Dunstan<andrew@dunslane.net>
wrote:Are we sure that's going to stop the DOS issue?
As long as it's done right, I don't see how it wouldn't.
There might be a cleaner way to do it, but after a moment's inspection
of the script, I'd be inclined to just hack GITCVS::updater->new() to
throw error if $module is neither "master" nor "REL\d_\d_STABLE".
Keep in mind of course that I'm a lousy Perl coder.Here's a quick change that will allow you to specifig a "modules" in
the gitcvs section to export, like:
[gitcvs]
enabled=1
modules=master,REL9_0_STABLE,REL8_4_STABLE--- git-cvsserver.orig 2010-09-23 12:03:06.000000000 -0400 +++ git-cvsserver 2010-09-23 13:16:53.000000000 -0400 @@ -2771,6 +2771,12 @@die "Git repo '$self->{git_path}' doesn't exist" unless ( -d
$self->{git_path} );+ if (defined $cfg->{gitcvs}{modules}) + { + $log->debug("Limitting modules: ". $cfg->{gitcvs}{modules}); + die "Invalid module $module" unless map {/^ *$module$/} split(',', $cfg->{gitcvs}{modules}); + } + $self->{dbdriver} = $cfg->{gitcvs}{$state->{method}}{dbdriver} || $cfg->{gitcvs}{dbdriver} || "SQLite"; $self->{dbname} = $cfg->{gitcvs}{$state->{method}}{dbname} ||OK, let's go with that. I was kinda hoping that we wouldn't have to do
this
at all, but Stefan has been having serious problems getting git to build
and
run on spoonbill, and I don't want to take up more of his time or be
without
it for very long.Are we doing this *just* for spoonbill? If so, it's a lot easier to
just filter-by-IP, so we don't have to maintain a patched version...That's the only one I know of - there could certainly be others - but if
we're going to support continued CVS use I want to be able to test it, and I
don't have a static IP address. Can you filter by name lookup? I have a
dyndns name.
No, filtering is by IP. ssh forwarding might work.
OTOH, this patch seems pretty small and simple to maintain.
True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 10/07/2010 10:11 AM, Magnus Hagander wrote:
OTOH, this patch seems pretty small and simple to maintain.
True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.
Why not just make a local copy somewhere else and patch and run that?
It's just a Perl script, no?
cheers
andrew
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan <andrew@dunslane.net> wrote:
On 10/07/2010 10:11 AM, Magnus Hagander wrote:
OTOH, this patch seems pretty small and simple to maintain.
True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.Why not just make a local copy somewhere else and patch and run that? It's
just a Perl script, no?
Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 10/07/2010 03:37 PM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net> wrote:
On 10/07/2010 10:11 AM, Magnus Hagander wrote:
OTOH, this patch seems pretty small and simple to maintain.
True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.Why not just make a local copy somewhere else and patch and run that? It's
just a Perl script, no?Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..
There's a simpler solution which I have just tested. Instead of
patching, use the Pg driver instead of SQLite. Set the dbname to %m. If
the database doesn't exist the cvs checkout will fail. So we just set up
databases for the modules we want to export (master and RELn_m_STABLE
for the live branches).
cheers
andrew
On 10/07/2010 09:52 PM, Andrew Dunstan wrote:
On 10/07/2010 03:37 PM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net>
wrote:On 10/07/2010 10:11 AM, Magnus Hagander wrote:
OTOH, this patch seems pretty small and simple to maintain.
True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.Why not just make a local copy somewhere else and patch and run
that? It's
just a Perl script, no?Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..There's a simpler solution which I have just tested. Instead of
patching, use the Pg driver instead of SQLite. Set the dbname to %m.
If the database doesn't exist the cvs checkout will fail. So we just
set up databases for the modules we want to export (master and
RELn_m_STABLE for the live branches).
BTW, because git-cvsserver treats a branch as a module, there needs to
be a small change in the buildfarm client to allow us to use it. I'm
working on that.
cheers
andrew
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan <andrew@dunslane.net> wrote:
On 10/07/2010 03:37 PM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net> wrote:
On 10/07/2010 10:11 AM, Magnus Hagander wrote:
<
OTOH, this patch seems pretty small and simple to maintain.True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.Why not just make a local copy somewhere else and patch and run that?
It's
just a Perl script, no?Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).
A database per branch seems like a horrible idea in general, but if it
works us around the bug, it seems like a doable idea.. As long as
we'll never have a branch called "postgres" or "git" (already in use
on that box).
I'll look into it.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 10/08/2010 02:09 AM, Magnus Hagander wrote:
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net> wrote:
There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).A database per branch seems like a horrible idea in general, but if it
works us around the bug, it seems like a doable idea.. As long as
we'll never have a branch called "postgres" or "git" (already in use
on that box).I'll look into it.
That's what the default SQLite setup does anyway. The only difference
here is that we are leveraging the fact that with the Pg driver the
database must first exist. Of course, with Pg the database can live on a
different host or in a server run on a different port, if you need to
avoid naming conflicts.
cheers
andrew
On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstan <andrew@dunslane.net> wrote:
On 10/08/2010 02:09 AM, Magnus Hagander wrote:
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net> wrote:
There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the
database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).
Wouldn't it be simpler be to generate hourly tarball on some host and wget it?
It can be generated even more often, as no history need to be kept.
Considering the state of cvsserver, can you be certain that whatever
is coming from it is really the most recent code?
--
marko
On 10/08/2010 09:15 AM, Marko Kreen wrote:
On Fri, Oct 8, 2010 at 3:13 PM, Andrew Dunstan<andrew@dunslane.net> wrote:
On 10/08/2010 02:09 AM, Magnus Hagander wrote:
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan<andrew@dunslane.net> wrote:
There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the
database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).Wouldn't it be simpler be to generate hourly tarball on some host and wget it?
It can be generated even more often, as no history need to be kept.Considering the state of cvsserver, can you be certain that whatever
is coming from it is really the most recent code?
Sure you can, why not? It will be coming from the same git repo that
servers git requests. git-cvsserver doesn't create a new CVS repo, it
emulates CVS from a git repo.
cheers
andrew
On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstan <andrew@dunslane.net> wrote:
That's what the default SQLite setup does anyway. The only difference here
is that we are leveraging the fact that with the Pg driver the database must
first exist. Of course, with Pg the database can live on a different host or
in a server run on a different port, if you need to avoid naming conflicts.
That can be done in SQLite to, just set the "database name" to a path
where there is no create access. Pre-create the database, and don't
let SQLite "create" new ones every time they are accessed. Standard
unix permissins should easily allow that setup. chmod -w on the
directory the database files go in.
a.
--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.
On 10/08/2010 10:53 AM, Aidan Van Dyk wrote:
On Fri, Oct 8, 2010 at 8:13 AM, Andrew Dunstan<andrew@dunslane.net> wrote:
That's what the default SQLite setup does anyway. The only difference here
is that we are leveraging the fact that with the Pg driver the database must
first exist. Of course, with Pg the database can live on a different host or
in a server run on a different port, if you need to avoid naming conflicts.That can be done in SQLite to, just set the "database name" to a path
where there is no create access. Pre-create the database, and don't
let SQLite "create" new ones every time they are accessed. Standard
unix permissins should easily allow that setup. chmod -w on the
directory the database files go in.
*shrug*. We are the PostgreSQL project after all :-) But whatever Magnus
wants to do is OK with me.
cheers
andrew
On Fri, Oct 8, 2010 at 08:09, Magnus Hagander <magnus@hagander.net> wrote:
On Fri, Oct 8, 2010 at 03:52, Andrew Dunstan <andrew@dunslane.net> wrote:
On 10/07/2010 03:37 PM, Magnus Hagander wrote:
On Thu, Oct 7, 2010 at 21:31, Andrew Dunstan<andrew@dunslane.net> wrote:
On 10/07/2010 10:11 AM, Magnus Hagander wrote:
<
OTOH, this patch seems pretty small and simple to maintain.True, it is rather small.
Does anybody know if there's an automated way to maintain that on
freebsd ports, and if so, how that works? I want to be *sure* we can't
accidentally upgrade git-cvsserver *without* the patch, since that is
a security issue.Why not just make a local copy somewhere else and patch and run that?
It's
just a Perl script, no?Yeah, but then we have to remember to manually patch that one when
somebody *else* finds/fixes a security issue. We have automatic
monitoring on the ports stuff to detect when that happens..There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).A database per branch seems like a horrible idea in general, but if it
works us around the bug, it seems like a doable idea.. As long as
we'll never have a branch called "postgres" or "git" (already in use
on that box).I'll look into it.
Should be up and working now. master branch is ready, working on
prepping rel9_0_stable.
CVSROOT is :pserver:anonymous@git.postgresql.org:/postgresql.git
module name is master or REL9_0_STABLE. master available now,
rel_9_0_stable sohuld be available in about half an hour.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On 10/12/2010 03:57 PM, Magnus Hagander wrote:
There's a simpler solution which I have just tested. Instead of patching,
use the Pg driver instead of SQLite. Set the dbname to %m. If the database
doesn't exist the cvs checkout will fail. So we just set up databases for
the modules we want to export (master and RELn_m_STABLE for the live
branches).A database per branch seems like a horrible idea in general, but if it
works us around the bug, it seems like a doable idea.. As long as
we'll never have a branch called "postgres" or "git" (already in use
on that box).I'll look into it.
Should be up and working now. master branch is ready, working on
prepping rel9_0_stable.CVSROOT is :pserver:anonymous@git.postgresql.org:/postgresql.git
module name is master or REL9_0_STABLE. master available now,
rel_9_0_stable sohuld be available in about half an hour.
Thanks for this.
I have tested it (see
<http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=quoll&dt=2010-10-12%2019:55:03>)
and there is a new Release of the buildfarm client to support its use:
see <http://pgfoundry.org/frs/?group_id=1000040>
cheers
andrew