Git cvsserver serious issue

Magnus Hagander <magnus@hagander.net> writes:

Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.

That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Ugh.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?

As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

regards, tom lane

On Wed, Sep 22, 2010 at 16:23, Tom Lane <tgl@sss.pgh.pa.us> wrote:

Magnus Hagander <magnus@hagander.net> writes:

Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.

That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Ugh.

Indeed.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?

As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address.  It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves.  I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

We could. It's currently on a freebsd vm though and I don't think we
can set per-server IP filters on those. (I was thinking iptables). We
could move it though - it doesn't *have* to be on the anonymous git
VM. It's just some extra resources.

Well, the use-case I was thinking of was Stefan. While he can't run
git on each and every animal, he certainly has *some* machine(s) on
the correct side of whatever firewall there may be that can run git.

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or not.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 09/22/2010 10:23 AM, Tom Lane wrote:

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

I'm not convinced we need any such thing yet. 13 of the 38 animals that
have reported in the last 5 days are using git already (OK, factoring
out my animals that's 8 out of 33). I'm going to send out email in a few
days prodding people to migrate. Let's see how far we get.

cheers

andrew

Andrew Dunstan <andrew@dunslane.net> writes:

On 09/22/2010 10:23 AM, Tom Lane wrote:

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

I'm not convinced we need any such thing yet. 13 of the 38 animals that
have reported in the last 5 days are using git already (OK, factoring
out my animals that's 8 out of 33). I'm going to send out email in a few
days prodding people to migrate. Let's see how far we get.

Even if we get 100% compliance on the buildfarm side, Peter's already
stated that moving the NLS support over to git is going to take more
time than we have available right now. We need a cvsserver for awhile
yet. We can't just suddenly announce "CVS service is terminated as of
yesterday" and expect that that's not going to have any serious
consequences.

Is there anything we could do to patch the problem out of git-cvsserver?
Maybe hack it to only accept requests for the active branch tips?

regards, tom lane

On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:

That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?

Well, let's see. If someone can figure out the git equivalent of

if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi

(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)

then I might be able to get this fixed.

On Wed, Sep 22, 2010 at 12:21 PM, Peter Eisentraut <peter_e@gmx.net> wrote:

On ons, 2010-09-22 at 16:03 +0200, Magnus Hagander wrote:

That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?

Well, let's see.  If someone can figure out the git equivalent of

if cvs -q update | egrep -q '^(U|P) '; then
 # ... something changed, so run the update ...
fi

(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)

then I might be able to get this fixed.

Can you just check whether the commit SHA of HEAD has changed? e.g.

git show-ref --heads -s master
git log --format=format:%H -n 1 master

...and compare with previous results of same?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

Excerpts from Peter Eisentraut's message of mié sep 22 12:21:45 -0400 2010:

Well, let's see. If someone can figure out the git equivalent of

if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi

(assuming, for simplicity, that the current directory has the
appropriate branch checked out already)

then I might be able to get this fixed.

Would it work to save the previous commit hash in a file and compare to
the current one?

--
Álvaro Herrera <alvherre@commandprompt.com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

At 2010-09-22 19:21:45 +0300, peter_e@gmx.net wrote:

Well, let's see. If someone can figure out the git equivalent of

if cvs -q update | egrep -q '^(U|P) '; then
# ... something changed, so run the update ...
fi

I think you want:

git pull
if [ $(git rev-parse HEAD) != $(git rev-parse ORIG_HEAD) ]; then
# ... the pull changed something ...
fi

-- ams

On 09/22/2010 10:26 AM, Magnus Hagander wrote:

On Wed, Sep 22, 2010 at 16:23, Tom Lane<tgl@sss.pgh.pa.us> wrote:

Magnus Hagander<magnus@hagander.net> writes:

Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.
That basically means that git-cvsserver is completely useless in a
public scenario as it stands. An easier way to DOS our server is hard
to find, really.

Ugh.

Indeed.

Now, if we can limit this by IP address, that would be ok. I assume we
can do this for the NLS stuff - peter?
As for buildfarm members needing CVS - is it workable to require that
the maintainers of these set up their own git clone with git cvsserver
(over ssh or pserver) and restrict it locally to the IP(s) of their
machines?

If we're going to let people in by IP address, maybe we could let legacy
buildfarm members in by IP address. It doesn't seem particularly
helpful to expect each buildfarm owner to solve this problem for
themselves. I'd also note that if they could run git locally, they
wouldn't be needing cvsserver in the first place.

We could. It's currently on a freebsd vm though and I don't think we
can set per-server IP filters on those. (I was thinking iptables). We
could move it though - it doesn't *have* to be on the anonymous git
VM. It's just some extra resources.

Well, the use-case I was thinking of was Stefan. While he can't run
git on each and every animal, he certainly has *some* machine(s) on
the correct side of whatever firewall there may be that can run git.

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or not.

Thinking about this some more, how about we do non-anonymous CVS over
SSH access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?

I'm not sure if that would handle other requirements, such as Peter's,
but I hope the residual requirements for CVS support will be pretty rare.

cheers

andrew

On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan <andrew@dunslane.net> wrote:

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or
not.

Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?

Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).

I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...

I'm not sure if that would handle other requirements, such as Peter's, but I
hope the residual requirements for CVS support will be pretty rare.

Just to be sure - do we have any other requirements for CVS *beyond*
buildfarm and NLS that we're not thinking of here?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 09/23/2010 02:09 AM, Magnus Hagander wrote:

On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net> wrote:

Also, couldn't we just set up the cvsserver on its own VM with a limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or
not.

Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?

Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).

I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...

Yes, I think we should have a VM. Is that so hard to do in these days of
Xen etc? I'm surprised we can't run up a VM pretty much at the drop of a
hat.

I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can
be sniffed or attacked using brute force. IP addresses can be spoofed.
But you're the SA, not me.

cheers

andrew

On Thu, Sep 23, 2010 at 11:27, Andrew Dunstan <andrew@dunslane.net> wrote:

On 09/23/2010 02:09 AM, Magnus Hagander wrote:

On Thu, Sep 23, 2010 at 04:59, Andrew Dunstan<andrew@dunslane.net>  wrote:

Also, couldn't we just set up the cvsserver on its own VM with a
limited
amount of disk space, and not worry too much about any "DOS threat"?
If somebody does do this, block them and reinitialize that server.

We could do that, but that could end up fighting a losing battle in
case some bot hits it.

I don't like deploying something with a known issue on it, sandboxed or
not.

Thinking about this some more, how about we do non-anonymous CVS over SSH
access to the git-cvsserver for the few buildfarm members that can't
currently handle using git (e.g. spoonbill)?

Well, if we do that centrally, we are back to a dedicated VM (hint:
we're most certainly not adding non-personal no-password accounts to
one of the VMs used for critical services - it's bad enough we have
Bruce's account there :P).

I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...

Yes, I think we should have a VM. Is that so hard to do in these days of Xen
etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.

In the new infrastructure, it is.

The main resource that's limited really is IP addresses ;)

I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can be
sniffed or attacked using brute force. IP addresses can be spoofed. But
you're the SA, not me.

I prefer keys. But I don't want those users on the same VM as
important services, because passphrase-less keys are a lot more likely
to be compromised than the keys of say a regular committer...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:

I assume most buildfarm clients are off static IPs (at least as seen
from the servers - they may be behind a NAT device, but that one
having static out)? If so, it seems simply easier to use pserver...

Yes, I think we should have a VM. Is that so hard to do in these days of Xen
etc? I'm surprised we can't run up a VM pretty much at the drop of a hat.

In the new infrastructure, it is.

The main resource that's limited really is IP addresses ;)

I was suggesting that the accounts would be protected using ssh keys.
Password and IP address protection seem pretty weak to me. Passwords can be
sniffed or attacked using brute force. IP addresses can be spoofed. But
you're the SA, not me.

I prefer keys. But I don't want those users on the same VM as
important services, because passphrase-less keys are a lot more likely
to be compromised than the keys of say a regular committer...

Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On 09/23/2010 09:55 AM, Bruce Momjian wrote:

Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?

Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.

cheers

andrew

Andrew Dunstan wrote:

On 09/23/2010 09:55 AM, Bruce Momjian wrote:

Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?

Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.

I suppose there is no way to pull diffs from git continuously and commit
them to CVS.

--
Bruce Momjian <bruce@momjian.us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ It's impossible for everything to be true. +

On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:

Andrew Dunstan wrote:

On 09/23/2010 09:55 AM, Bruce Momjian wrote:

Stupid question, but can't we just create a CVSROOT fed from git, and
use the normal CVS server to feed sites?

Where is it going to get the ,v files that CVS uses from? git-cvsserver
emulates a CVS server from git. It doesn't create a CVS repository that
CVS in server mode could use.

I suppose there is no way to pull diffs from git continuously and commit
them to CVS.

You probably could. I'm very doubtful it's worth it...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On Thu, Sep 23, 2010 at 04:38:27PM +0200, Magnus Hagander wrote:

On Thu, Sep 23, 2010 at 16:11, Bruce Momjian <bruce@momjian.us> wrote:

Andrew Dunstan wrote:

On 09/23/2010 09:55 AM, Bruce Momjian wrote:

Stupid question, but can't we just create a CVSROOT fed from
git, and use the normal CVS server to feed sites?

Where is it going to get the ,v files that CVS uses from?
git-cvsserver emulates a CVS server from git. It doesn't create a
CVS repository that CVS in server mode could use.

I suppose there is no way to pull diffs from git continuously and
commit them to CVS.

You probably could. I'm very doubtful it's worth it...

Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?

Cheers,
David.
--
David Fetter <david@fetter.org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fetter@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Magnus Hagander <magnus@hagander.net> writes:

So, I found (with some helpful hints from Robert who caught the final
nail in the coffin) a good reason why we really can't run a
git-cvsserver globally.

Any user can point their cvs client at the repository. And check out
an arbitrary branch, tag *or individual commit*. Doing so will create
a 50Mb sqlite database on the server with cache information about that
head.

I'm still wondering why we don't simply lobotomize git-cvsserver to
refuse requests to check out anything except the active branch tips.
It's only a Perl script. I could probably hack it in an hour,
there are those here who could do it in ten minutes.

regards, tom lane

On 09/23/2010 10:58 AM, David Fetter wrote:

Back to a question you asked earlier, what exactly still depends on
CVS right now, as in which buildfarm animals, what parts of the NLS
processes? Also as you asked earlier, what else?

At least one buildfarm member, spoonbill, is known to have issues with
git. This machine, because of the flags it uses, has found numerous bugs
for us in the past and is quite important for us to maintain (a direct
counter argument to your suggestion of dropping platforms that don't
support git).

cheers

andrew