CVS repository invalid revision

Christian Robottom Reis wrote:

I'm running a conversion of PostgreSQL's CVS repository, but I'm
stuck on a revision that cscvs fails to parse.

I can't quite answer that particular question, but there are already somewhat
stable conversions of the CVS repository to Subversion and Git available, so
perhaps you want to work off that.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Sat, Feb 16, 2008 at 08:28:42PM +0100, Peter Eisentraut wrote:

Christian Robottom Reis wrote:

I'm running a conversion of PostgreSQL's CVS repository, but I'm
stuck on a revision that cscvs fails to parse.

I can't quite answer that particular question, but there are already somewhat
stable conversions of the CVS repository to Subversion and Git available, so
perhaps you want to work off that.

Ah, interesting. Do you have a URL for the Subversion conversion? Is it
a mirror of CVS, and if so, how often is it updated?
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3376 0125

Christian Robottom Reis wrote:

I'm running a conversion of PostgreSQL's CVS repository, but I'm
stuck on a revision that cscvs fails to parse. The hint that the error
gives me is:

Parser error: failed to parse revision data line (line: 'date: 2000/12/04 01:20:38; author: tgl; state: Exp; lines:
')

That's the literal output -- I'm not sure what the linebreak after
"lines:" means.

Can someone help me out by finding and fixing the corrupted revision?
I"d be most grateful.

There's no corrupted revision -- the text you see is part of a log
message, not a real header line. So the cscvs tool would seem to
need to be able to cope with that. The complete entry (this is from
contrib/pgcrypto/md5.c, but there are several more files touched by this
commit) is this:

revision 1.3
date: 2001-01-09 13:07:13 -0300; author: momjian; state: Exp; lines: +16 -16;
The KAME files md5.* and sha1.* have the following changelog
entry:

----------------------------
revision 1.2
date: 2000/12/04 01:20:38; author: tgl; state: Exp; lines:
+18 -18
Eliminate some of the more blatant platform-dependencies ... it
builds here now, anyway ...
----------------------------

Which basically changes u_int*_t -> uint*_t, so now it does not
compile neither under Debian 2.2 nor under NetBSD 1.5 which
is platform independent<B8> all right. Also it replaces $KAME$
with $Id$ which is Bad Thing. PostgreSQL Id should be added as a
separate line so the file history could be seen.

So here is patch:

* changes uint*_t -> uint*. I guess that was the original
intention
* adds uint64 type to include/c.h because its needed
[somebody should check if I did it right]
* adds back KAME Id, because KAME is the master repository
* removes stupid c++ comments in pgcrypto.c
* removes <sys/types.h> from the code, its not needed

--
marko

Marko Kreen

And then you have the actual 1.2 revision mentioned in the log message,
which is

revision 1.2
date: 2000-12-03 22:20:38 -0300; author: tgl; state: Exp; lines: +18 -18;
Eliminate some of the more blatant platform-dependencies ... it builds here now, anyway ...

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Sun, 17 Feb 2008, Christian Robottom Reis wrote:

Ah, interesting. Do you have a URL for the Subversion conversion? Is it
a mirror of CVS, and if so, how often is it updated?

The Subversion one is at
https://projects.commandprompt.com/public/pgsql/repo/ and I haven't
noticed comments on how often that particular one updates yet.

On a broader note, there is a list of alternate repositories available
with notes about their update frequency at
http://developer.postgresql.org/index.php/Working_with_CVS#Other_versions_of_the_PostgreSQL_Repository
that I try to keep up to date as people mention new ones.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 18 Feb 2008 15:59:29 -0500 (EST)
Greg Smith <gsmith@gregsmith.com> wrote:

On Sun, 17 Feb 2008, Christian Robottom Reis wrote:

Ah, interesting. Do you have a URL for the Subversion conversion?
Is it a mirror of CVS, and if so, how often is it updated?

The Subversion one is at
https://projects.commandprompt.com/public/pgsql/repo/ and I haven't
noticed comments on how often that particular one updates yet.

I think it is every 4 hours but I am in the process of changing it
(after testing) so that it is every 10 minutes.

Joshua D. Drake

- --
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHuxivATb/zqfZUUQRAmwiAJ9esKdMzhqiJeCR712A2HirymZocQCcDqez
KbrSmqmstQc52NDqJZNziBA=
=2TVA
-----END PGP SIGNATURE-----

On Mon, Feb 18, 2008 at 03:13:24PM -0300, Alvaro Herrera wrote:

Christian Robottom Reis wrote:

I'm running a conversion of PostgreSQL's CVS repository, but I'm
stuck on a revision that cscvs fails to parse. The hint that the error
gives me is:

Parser error: failed to parse revision data line (line: 'date: 2000/12/04 01:20:38; author: tgl; state: Exp; lines:
')

That's the literal output -- I'm not sure what the linebreak after
"lines:" means.

Can someone help me out by finding and fixing the corrupted revision?
I"d be most grateful.

There's no corrupted revision -- the text you see is part of a log
message, not a real header line. So the cscvs tool would seem to
need to be able to cope with that. The complete entry (this is from
contrib/pgcrypto/md5.c, but there are several more files touched by this
commit) is this:

revision 1.3
date: 2001-01-09 13:07:13 -0300; author: momjian; state: Exp; lines: +16 -16;
The KAME files md5.* and sha1.* have the following changelog
entry:

----------------------------
revision 1.2
date: 2000/12/04 01:20:38; author: tgl; state: Exp; lines:
+18 -18
Eliminate some of the more blatant platform-dependencies ... it
builds here now, anyway ...
----------------------------

Wow, that's very interesting. Thanks for pointing it out; now that I
have this in hand, I need to chase somebody interested in fixing a cscvs
bug <wink>
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3376 0125

Christian Robottom Reis <kiko@canonical.com> writes:

On Mon, Feb 18, 2008 at 03:13:24PM -0300, Alvaro Herrera wrote:

There's no corrupted revision -- the text you see is part of a log
message, not a real header line. So the cscvs tool would seem to
need to be able to cope with that.

Wow, that's very interesting. Thanks for pointing it out; now that I
have this in hand, I need to chase somebody interested in fixing a cscvs
bug <wink>

I notice that cvsweb doesn't handle it very gracefully, either.

It looks to me like it's just about impossible for anything that
is parsing "cvs log" output to tell the difference between this and
a genuine revision entry header, which means it's probably breaking
most of the other conversion tools too.

Perhaps it'd be worth hacking the CVS repository entries to modify
these log entries a bit? Indenting the quoted revision entry a few
spaces would probably do it.

According to
http://archives.postgresql.org/pgsql-committers/2001-01/msg00115.php
the affected files are contrib/pgcrypto/
md5.c md5.h pgcrypto.c pgcrypto.h sha1.c sha1.h

regards, tom lane

Being prodded by this, I decided to take another look at why cvsps
doesn't work on PostgreSQL CVS repository...

It happens that it's choking on a bunch of the various REL7_1 tags...
And, the fromcvs converter, which works fine, just happens to *not*
support importing tags - go figure ;-)

So, my solution - strip them all out, and low-and-behold, cvsps works
(and thus, so does git-cvsimport).

So, for people having problems with the "cvs repository", try and
eliminate the REL7_1* tags, and see if the problems go away...

Here's my ugly 30 second perl to "remove" the REL7_* tags...

Of course, git-cvsimport (with tags stripped) takes hours to convert the
repository, rather than the couple 10s of minutes for fromcvs, but at
least it works...

a.

--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.

On Wed, Feb 20, 2008 at 05:00:14PM -0500, Aidan Van Dyk wrote:

So, for people having problems with the "cvs repository", try and
eliminate the REL7_1* tags, and see if the problems go away...

I'm currently trying an import off Subversion. Is there a copy of the
CVS repo somewhere I can rsync, if that doesn't work?
--
Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3376 0125

Aidan Van Dyk wrote:

And, the fromcvs converter, which works fine, just happens to *not*
support importing tags - go figure ;-)

Meaning it chokes on the $PostgreSQL: ...$ stuff? or what?

cheers

andrew

Christian Robottom Reis wrote:

I'm currently trying an import off Subversion. Is there a copy of the
CVS repo somewhere I can rsync, if that doesn't work?

Sure:

http://www.postgresql.org/docs/8.3/interactive/rsync.html

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

* Andrew Dunstan <andrew@dunslane.net> [080220 17:11]:

Aidan Van Dyk wrote:

And, the fromcvs converter, which works fine, just happens to *not*
support importing tags - go figure ;-)

Meaning it chokes on the $PostgreSQL: ...$ stuff? or what?

$something$ are "keywords" in CVS speak. Tags are "cvs tag" type
operations that put "human labels" on a specific set of files (like
REL7_1BETA2, etc).

It's the cvs tags, not keywords that are causing cvsps the pain
(and thus git cvsimport).

fromcvs doesn't try and import tags at all, so it happens to avoid that
that tag problem completely.

I haven't tried to figure out exactly what tag it is, or what file(s)
the tags are problematic on yet, because it takes repeated munging/cvsps
runs to figure out, and I haven' had the time yet... But just blindkly
removing all the REL7_1* tags "worked for me"...

a.

--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.

Christian Robottom Reis wrote:

On Wed, Feb 20, 2008 at 05:00:14PM -0500, Aidan Van Dyk wrote:

So, for people having problems with the "cvs repository", try and
eliminate the REL7_1* tags, and see if the problems go away...

I'm currently trying an import off Subversion. Is there a copy of the
CVS repo somewhere I can rsync, if that doesn't work?

rsync anoncvs.postgresql.org::pgsql-cvs /path/to/mirror

cheers

andrew

* Christian Robottom Reis <kiko@canonical.com> [080220 17:32]:

On Wed, Feb 20, 2008 at 05:00:14PM -0500, Aidan Van Dyk wrote:

So, for people having problems with the "cvs repository", try and
eliminate the REL7_1* tags, and see if the problems go away...

I'm currently trying an import off Subversion. Is there a copy of the
CVS repo somewhere I can rsync, if that doesn't work?

anoncvs allows rsync:
rsync -qavzCH --delete anoncvs.postgresql.org::pgsql-cvs PostgreSQL/

--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.

* Aidan Van Dyk <aidan@highrise.ca> [080220 17:00]:

It happens that it's choking on a bunch of the various REL7_1 tags...
And, the fromcvs converter, which works fine, just happens to *not*
support importing tags - go figure ;-)

So, my solution - strip them all out, and low-and-behold, cvsps works
(and thus, so does git-cvsimport).

So, for people having problems with the "cvs repository", try and
eliminate the REL7_1* tags, and see if the problems go away...

Here's my ugly 30 second perl to "remove" the REL7_* tags...

I see I forgot to include it.

I've found that it's only REL7_1_BETA[23] that cause the problem...

And it seems like both those tags are taging various files from various
points in time, and such a way that cvsps loops forever...

Here' my perl to "fix" the repository and allow cvsps to work:

It's ugly, and I make no guarantee that it doesn't "corrupt" some binary
file somewhere...

#!/usr/bin/perl
my @TAGS = qw(REL7_1_BETA2 REL7_1_BETA3);
for my $file (@ARGV)
{
my $i, $o;
my @sb = stat($file);
printf "%s: %d\n", $file, $sb[7];

open(INPUT, "<:raw", $file) || die "Couldn't open $file: $!\n";
$i = sysread INPUT, $buf, $sb[7];
close INPUT;

for my $tag (@TAGS)
{
$buf =~ s/ $tag:[.0-9]*\n//es;
$buf =~ s/\n $tag:[.0-9]*;\n/;\n/es;
}

open OUTPUT, ">:raw", "$file.new";
$o = syswrite OUTPUT, $buf;
close OUTPUT;
printf "IN: %d OUT: %d\n", $i, $o;
rename ("$file.new", $file) || die "Couldn't rename: $!\n";
}

Run it over the repository like:
find CVSROOT/pgsql -name \*,v | xargs perl parse.pl

YMMV...

a.

--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.