[SUGGESTION] CVSync
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I tried
to use rsync because cvsup is simply not an option for me.
I'd like to suggest that CVSync (www.cvsync.org) be added as a means for
the public to obtain a local mirror of the cvs repository. It builds on
most every platform (it's written in C, no modula-3 required) and has the
advantage of caching directory scan info to reduce server load -- The
cached scan info is sent to the client, which compares it to its own
cached scan info to decide what to download (You can set up a cron job to
periodically scan the repository directories, or even set up a
sophisticated system that would only scan after commits are made.). It
also sends only the new diffs (optionally compressed) from the ",v" files
that are needed to bring the clients' repository mirror up to date.
Thanks!
Joel
Joel Miller wrote:
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I
tried to use rsync because cvsup is simply not an option for me.I'd like to suggest that CVSync (www.cvsync.org) be added as a means
for the public to obtain a local mirror of the cvs repository. It
builds on most every platform (it's written in C, no modula-3
required) and has the advantage of caching directory scan info to
reduce server load -- The cached scan info is sent to the client,
which compares it to its own cached scan info to decide what to
download (You can set up a cron job to periodically scan the
repository directories, or even set up a sophisticated system that
would only scan after commits are made.). It also sends only the new
diffs (optionally compressed) from the ",v" files that are needed to
bring the clients' repository mirror up to date.
As a complete out of nowhere suggestion, you might also look
at csup in FreeBSD's CVS (or the project page), which is a cvsup clone
in C.
LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 512-248-2683 E-Mail: ler@lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I tried
to use rsync because cvsup is simply not an option for me.
Any particular reason why straight CVS doesn't work for you? Are you
that interested in having the log comment locally?
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
Show quoted text
Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
tool for doing 5% of the work and then sitting around waiting for someone
else to do the other 95% so you can sue them.
Martijn van Oosterhout <kleptog@svana.org> writes:
Any particular reason why straight CVS doesn't work for you? Are you
that interested in having the log comment locally?
Personally, I'd really like to have a local repository copy, because
I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
several standard deviations away from the mean. So far I've been
discouraged from setting up a repository by the unreasonable
infrastructure needs of cvsup. So these alternatives do sound pretty
interesting.
Is csup protocol-compatible with cvsup? If so people could use it
without Marc having to do anything. Has anyone got experience with
it --- reliability, performance, etc?
regards, tom lane
Tom Lane wrote:
Martijn van Oosterhout <kleptog@svana.org> writes:
Any particular reason why straight CVS doesn't work for you? Are you
that interested in having the log comment locally?Personally, I'd really like to have a local repository copy, because
I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
several standard deviations away from the mean. So far I've been
discouraged from setting up a repository by the unreasonable
infrastructure needs of cvsup. So these alternatives do sound pretty
interesting.Is csup protocol-compatible with cvsup? If so people could use it
without Marc having to do anything. Has anyone got experience with
it --- reliability, performance, etc?
Tom,
I don't know what unreasonable infrastructure you are referring to.
Building cvsup is a major pain, but installing and running it isn't, in
my experience. There's a package in Fedora Extras. Setting up cvsweb
against my cvsup repo is a fine idea - I wonder why I didn't think of that.
That's not to say that supporting cvsync isn't a good idea too.
TIMTOWTDI as we perl people like to say.
cheers
andrew
It's only partially compatible: Like I mentioned earlier, csup currently
only acts as an alternative cvs checkout client.
Joel
On Thu, Mar 23, 2006 at 3:15 PM, Tom Lane (tgl@sss.pgh.pa.us) wrote:
Show quoted text
Is csup protocol-compatible with cvsup? If so people could use it
without Marc having to do anything. Has anyone got experience with
it --- reliability, performance, etc?
Import Notes
Resolved by subject fallback
Plain old cvs doesn't work for me because I need to get the repository,
not just a checkout of the sources, so I can do diffs, browse old
versions and what-not. I find that it's easier to understand code that's
been around for a while if I can observe how it's changed over time.
Usually, when I have time to these things, it also happens that I am
offline without access to a net connection.
Thanks!
Joel
On Thu, Mar 23, 2006 at 2:50 PM, Martijn van Oosterhout
(kleptog@svana.org) wrote:
Show quoted text
Any particular reason why straight CVS doesn't work for you? Are you
that interested in having the log comment locally?
On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I tried
to use rsync because cvsup is simply not an option for me.
Which seems to be a permission problem to me. Maybe the
permissions should just get changed so that you can use it using
rsync?
I also find it very handy to have a local mirror of the
repository.
Kurt
Yes, that's one way to solve the problem. I was just trying to suggest
something that would benefit everyone in general while not opening up a
can of worms with regard to increased server load in the long term.
Joel
On Fri, Mar 24, 2006 at 4:34 PM, Kurt Roeckx (kurt@roeckx.be) wrote:
Show quoted text
On Thu, Mar 23, 2006 at 01:32:34PM -0800, Joel Miller wrote:
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I tried
to use rsync because cvsup is simply not an option for me.Which seems to be a permission problem to me. Maybe the
permissions should just get changed so that you can use it using
rsync?I also find it very handy to have a local mirror of the
repository.Kurt
[I thought this had gone out to the list when I first sent it, but now I
see that it didn't]
Actually, I've already looked at it very closely (spent a whole day
browsing the latest source code, in fact). The problem is csup still
doesn't support mirroring of the repository ",v" files. It only acts as
an alternative cvs checkout client. It doesn't handle ",v" files at all.
The main developer of csup is looking for help in implementing support
for dealing with the RCS ",v" files but so far it looks like no one is
offering. And I don't think anyone is going to, since cvsync is already
up and running, and so easy to set up and use.
Joel
On Thu, Mar 23, 2006 at 1:59 PM, Larry Rosenman (ler@lerctr.org) wrote:
Show quoted text
Joel Miller wrote:
Hello,
I've been trying to get a local mirror of the cvs repository, but my
connection attempts using rsync to cvsup.postgresql.org are always
refused when I try to actually retrieve the pgsql-cvs collection. I
tried to use rsync because cvsup is simply not an option for me.I'd like to suggest that CVSync (www.cvsync.org) be added as a means
for the public to obtain a local mirror of the cvs repository. It
builds on most every platform (it's written in C, no modula-3
required) and has the advantage of caching directory scan info to
reduce server load -- The cached scan info is sent to the client,
which compares it to its own cached scan info to decide what to
download (You can set up a cron job to periodically scan the
repository directories, or even set up a sophisticated system that
would only scan after commits are made.). It also sends only the new
diffs (optionally compressed) from the ",v" files that are needed to
bring the clients' repository mirror up to date.As a complete out of nowhere suggestion, you might also look
at csup in FreeBSD's CVS (or the project page), which is a cvsup clone
in C.LER
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 512-248-2683 E-Mail: ler@lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Import Notes
Resolved by subject fallback
On Thu, 2006-03-23 at 18:15 -0500, Tom Lane wrote:
Personally, I'd really like to have a local repository copy, because
I spend a *lot* of time with cvsweb etc --- but I'm sure my needs are
several standard deviations away from the mean.
I'm actually amazed that anyone does any serious amount of work on
Postgres *without* using cvsup :)
I agree that investigating alternatives would be a good idea: AFAIK
there's no easy way to build cvsup on Linux/AMD64 (without patches and
more pain than I'm willing to endure), so I use cvsup on one machine and
then periodically rsync a copy of that to my main (AMD64) machine.
-Neil
Neil Conway <neilc@samurai.com> writes:
I agree that investigating alternatives would be a good idea: AFAIK
there's no easy way to build cvsup on Linux/AMD64 (without patches and
more pain than I'm willing to endure), so I use cvsup on one machine and
then periodically rsync a copy of that to my main (AMD64) machine.
Actually, the last time I looked seriously at cvsup, my requirement was
that it build on HPUX, which it did not (well, maybe it would've with
sufficient investment of pain, but that wasn't happening). I was about
to take another look at whether I couldn't just-install-it on FC4, but
my main FC4 machine is now x86_64, so your remarks are discouraging me
again :-(
This whole discussion reminds me why we've stuck so fervently to
bog-standard ANSI C for Postgres. There is a payoff for taking
portability seriously. Too bad the original authors of cvsup were more
interested in using a flavor-of-the-month programming language...
regards, tom lane
This whole discussion reminds me why we've stuck so fervently to
bog-standard ANSI C for Postgres. There is a payoff for taking
portability seriously. Too bad the original authors of cvsup were more
interested in using a flavor-of-the-month programming language...
Is there any reason why we don't allow rsync access to the cvs repo?
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
Tom Lane wrote:
Neil Conway <neilc@samurai.com> writes:
I agree that investigating alternatives would be a good idea: AFAIK
there's no easy way to build cvsup on Linux/AMD64 (without patches and
more pain than I'm willing to endure), so I use cvsup on one machine and
then periodically rsync a copy of that to my main (AMD64) machine.Actually, the last time I looked seriously at cvsup, my requirement was
that it build on HPUX, which it did not (well, maybe it would've with
sufficient investment of pain, but that wasn't happening). I was about
to take another look at whether I couldn't just-install-it on FC4, but
my main FC4 machine is now x86_64, so your remarks are discouraging me
again :-(
FWIW I use the i386 FC4 RPMs on my x86_64 FC4 machine -- works fine.
Joe
"Joshua D. Drake" <jd@commandprompt.com> writes:
Is there any reason why we don't allow rsync access to the cvs repo?
That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
mirror seems like it couldn't pose any serious security threat.
Marc?
regards, tom lane
Joshua D. Drake wrote:
Is there any reason why we don't allow rsync access to the cvs repo?
The only reason I can possibly think of is server load, comparing the timestamps for every file and directory in a repository for every rsync session could be taxing if everyone under the sun and all their cousins could do it.
CVSync, on the other hand, can cache those timestamps into a file and send it to the client, offloading the comparison workload from the server to the client where it belongs.
Joel
--
// file: mice.h
#include "frickin_lasers.h"
Tom Lane wrote:
"Joshua D. Drake" <jd@commandprompt.com> writes:
Is there any reason why we don't allow rsync access to the cvs repo?
That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
mirror seems like it couldn't pose any serious security threat.
Marc?
Well it would sure seem to eliminate alot of the issues :)
Joshua D. Drake
regards, tom lane
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
On Fri, 24 Mar 2006, Tom Lane wrote:
"Joshua D. Drake" <jd@commandprompt.com> writes:
Is there any reason why we don't allow rsync access to the cvs repo?
That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
mirror seems like it couldn't pose any serious security threat.
Marc?
I can't think of any reason *not* to .. if someone can, I can easily
disable, but for now ...
/usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On Fri, 24 Mar 2006, Joel Miller wrote:
Joshua D. Drake wrote:
Is there any reason why we don't allow rsync access to the cvs repo?
The only reason I can possibly think of is server load, comparing the
timestamps for every file and directory in a repository for every rsync
session could be taxing if everyone under the sun and all their cousins could
do it.CVSync, on the other hand, can cache those timestamps into a file and send it
to the client, offloading the comparison workload from the server to the
client where it belongs.
I'm going to look at installing this also ... what does the client side
need?
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Marc G. Fournier wrote:
On Fri, 24 Mar 2006, Tom Lane wrote:
"Joshua D. Drake" <jd@commandprompt.com> writes:
Is there any reason why we don't allow rsync access to the cvs repo?
That'd be Marc's bailiwick not mine ... but rsync from the anoncvs
mirror seems like it couldn't pose any serious security threat.
Marc?I can't think of any reason *not* to .. if someone can, I can easily
disable, but for now .../usr/local/bin/rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs .
This appears to work :), although everyone please note the
destination... you probably don't want the "," ;)
Joshua D. Drake
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
--
=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/
