Mirrors not tracking main ftp site?

On Mon, 4 Jun 2001, Tom Lane wrote:

However, it seems that the mirrors have a lot more stuff:
ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
of files back to 7.1beta6, and so do the other several I checked in
a random sample. Is the update mechanism failing to cause old files
to be removed from the mirrors?

I'm doing:

/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
/mnt/ftpd/pub/postgresql

every 4 hours and still have these files too. Not really sure where the
files are comming from, but when I deleted a file from /dev it didn't
come back on a sync.

I will look into the rsync info a bit more, but...

- Brandon

----------------------------------------------------------------------------
b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

However, it seems that the mirrors have a lot more stuff:
ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
of files back to 7.1beta6, and so do the other several I checked in
a random sample. Is the update mechanism failing to cause old files
to be removed from the mirrors?

Found the problem. Since rsync gets a perms denied from .hidden, it
refuses to delete files.

root@seraph:/root# ./rsync-postgres-ftp
receiving file list ... opendir(.hidden): Permission denied
done
IO error encountered - skipping file deletion
wrote 105 bytes read 20762 bytes 2782.27 bytes/sec
total size is 521221478 speedup is 24978.27

When I changed the script to:

#/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
/mnt/ftpd/pub/postgresql
/usr/local/bin/rsync -avz --ignore-errors --delete
hub.org::postgresql-ftp /mnt/ftpd/pub/postgresql

It worked. People need to either use the --ignore-errors or have the
.hidden folder on the server removed.

- - Brandon

- ----------------------------------------------------------------------------
b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBOxxDFPYgmKoG+YbuEQLeiACeIhRJQ0HTZQCJc+aqHzqSfTods7IAnjEO
m9vtW2WRh3PMPXdlWeEBzTzY
=u6ep
-----END PGP SIGNATURE-----

Tom Lane wrote:

which agrees with the view at http://www.ca.postgresql.org/ftpsite/dev/.

Thanks for the pointer to ftp-sites http url ;)

Now that I was able to verify (after not getting in to ftp:// for days)
that there are no 7.1.2 RPMS, I would like to inquire, if there are any
plans to make RPMS's.

----------------
Hannu

bpalmer <bpalmer@crimelabs.net> writes:

Found the problem. Since rsync gets a perms denied from .hidden, it
refuses to delete files.

Ah-hah. And that directory seems to have appeared on 13 Apr, which is
right about the time that the oldest un-deleted files on the mirrors are
from:

ls -ld ~pgsql/ftp/pub/.hidden

d--x--x--x 2 root pgsql 512 Apr 13 14:58 /home/projects/pgsql/ftp/pub/.hidden

Marc, what is that thing? Can we get rid of it?

It worked. People need to either use the --ignore-errors or have the
.hidden folder on the server removed.

I don't much care for the first alternative ...

regards, tom lane

On Mon, 4 Jun 2001, Tom Lane wrote:

On hub, in /home/projects/pgsql/ftp/pub/dev I see

*.tar.gz.md5 postgresql-opt-snapshot.tar.gz
doc postgresql-opt-snapshot.tar.gz.md5
postgresql-base-snapshot.tar.gz postgresql-snapshot.tar.gz
postgresql-base-snapshot.tar.gz.md5 postgresql-snapshot.tar.gz.md5
postgresql-docs-snapshot.tar.gz postgresql-test-snapshot.tar.gz
postgresql-docs-snapshot.tar.gz.md5 postgresql-test-snapshot.tar.gz.md5

which agrees with the view at http://www.ca.postgresql.org/ftpsite/dev/.

However, it seems that the mirrors have a lot more stuff:
ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
of files back to 7.1beta6, and so do the other several I checked in
a random sample. Is the update mechanism failing to cause old files
to be removed from the mirrors?

Here's the syntax we tell them to use:

rsync -avz --delete hub.org::[remote]/ [destination directory]

If that's not what they're using I can't go into their cronjobs and
fix it.

Also, some of the mirrors claimed to be up-to-date by
http://www.postgresql.org/index.html aren't. Fr instance,
download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.
I thought that the up-to-date check was automated?

It is and here's the directory from sourceforge:

227 Entering Passive Mode (64,28,67,101,18,128).
150 Opening ASCII mode data connection for file list
-rw-r--r-- 1 root root 8117016 May 24 16:37 postgresql-7.1.2.tar.gz
-rw-r--r-- 1 root root 65 May 24 16:38 postgresql-7.1.2.tar.gz.md5
-rw-r--r-- 1 root root 3240364 May 24 16:38 postgresql-base-7.1.2.tar.gz
-rw-r--r-- 1 root root 70 May 24 16:38 postgresql-base-7.1.2.tar.gz.md5
-rw-r--r-- 1 root root 2072096 May 24 16:38 postgresql-docs-7.1.2.tar.gz
-rw-r--r-- 1 root root 70 May 24 16:38 postgresql-docs-7.1.2.tar.gz.md5
-rw-r--r-- 1 root root 1803742 May 24 16:38 postgresql-opt-7.1.2.tar.gz
-rw-r--r-- 1 root root 69 May 24 16:38 postgresql-opt-7.1.2.tar.gz.md5
-rw-r--r-- 1 root root 1002166 May 24 16:38 postgresql-test-7.1.2.tar.gz
-rw-r--r-- 1 root root 70 May 24 16:38 postgresql-test-7.1.2.tar.gz.md5
226-Transfer complete.
226 Quotas off

What is it you find missing about 7.1.2? What were you actually looking
at?

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
Online Campground Directory http://www.camping-usa.com
Online Giftshop Superstore http://www.cloudninegifts.com
==========================================================================

okay, just removed the .hidden directory from the ftp server, which should
correct that ... I had setup that .hidden directory to be excluded though,
not sure why it was bothering things :(

On Mon, 4 Jun 2001, bpalmer wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

However, it seems that the mirrors have a lot more stuff:
ftp://postgresql.readysetnet.com/pub/postgresql/dev/ shows dozens
of files back to 7.1beta6, and so do the other several I checked in
a random sample. Is the update mechanism failing to cause old files
to be removed from the mirrors?

Found the problem. Since rsync gets a perms denied from .hidden, it
refuses to delete files.

root@seraph:/root# ./rsync-postgres-ftp
receiving file list ... opendir(.hidden): Permission denied
done
IO error encountered - skipping file deletion
wrote 105 bytes read 20762 bytes 2782.27 bytes/sec
total size is 521221478 speedup is 24978.27

When I changed the script to:

#/usr/local/bin/rsync -avz --delete hub.org::postgresql-ftp
/mnt/ftpd/pub/postgresql
/usr/local/bin/rsync -avz --ignore-errors --delete
hub.org::postgresql-ftp /mnt/ftpd/pub/postgresql

It worked. People need to either use the --ignore-errors or have the
.hidden folder on the server removed.

- - Brandon

- ----------------------------------------------------------------------------
b. palmer, bpalmer@crimelabs.net pgp:crimelabs.net/bpalmer.pgp5

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBOxxDFPYgmKoG+YbuEQLeiACeIhRJQ0HTZQCJc+aqHzqSfTods7IAnjEO
m9vtW2WRh3PMPXdlWeEBzTzY
=u6ep
-----END PGP SIGNATURE-----

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Marc G. Fournier ICQ#7615664 IRC Nick: Scrappy
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org

Vince Vielhaber <vev@michvhf.com> writes:

On Mon, 4 Jun 2001, Tom Lane wrote:

Also, some of the mirrors claimed to be up-to-date by
http://www.postgresql.org/index.html aren't. Fr instance,
download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.

What is it you find missing about 7.1.2? What were you actually looking
at?

I went to ftp://download.sourceforge.net/pub/mirrors/postgresql/
(the link given by our homepage) and didn't see the v7.1.2 symlink,
nor did the source subdirectory have a v7.1.2 subdirectory.

As of this morning, though, both are there. I suppose they synced up
overnight.

regards, tom lane

On Tue, 5 Jun 2001, Tom Lane wrote:

Vince Vielhaber <vev@michvhf.com> writes:

On Mon, 4 Jun 2001, Tom Lane wrote:

Also, some of the mirrors claimed to be up-to-date by
http://www.postgresql.org/index.html aren't. Fr instance,
download.sourceforge.net doesn't have 7.1.1 nor 7.1.2.

What is it you find missing about 7.1.2? What were you actually looking
at?

I went to ftp://download.sourceforge.net/pub/mirrors/postgresql/
(the link given by our homepage) and didn't see the v7.1.2 symlink,
nor did the source subdirectory have a v7.1.2 subdirectory.

As of this morning, though, both are there. I suppose they synced up
overnight.

Don't know what could have happened to it, I'm fairly certain I downloaded
it from them less than a week ago when I did some upgrading here.

Vince.
--
==========================================================================
Vince Vielhaber -- KA8CSH email: vev@michvhf.com http://www.pop4.net
56K Nationwide Dialup from $16.00/mo at Pop4 Networking
Online Campground Directory http://www.camping-usa.com
Online Giftshop Superstore http://www.cloudninegifts.com
==========================================================================