snapshot generation broken
Hi all!
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?
Stefan
On 22/09/10 11:33, Stefan Kaltenbrunner wrote:
Hi all!
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?
BTW, there is a nice command called in git to create tarballs:
git archive master | gzip > postgresql.tar.gz
I doubt we can use it because we add some generated files to our
tarballs that are not in the repository, but I thought I'd mention it.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
On Wed, Sep 22, 2010 at 10:47, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
On 22/09/10 11:33, Stefan Kaltenbrunner wrote:
Hi all!
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?BTW, there is a nice command called in git to create tarballs:
git archive master | gzip > postgresql.tar.gz
I doubt we can use it because we add some generated files to our tarballs
that are not in the repository, but I thought I'd mention it.
It is useful, yes, but not for this.
I'll take a peek at the tarball generation today, unless beaten to it
(meaning if someone wants to star tlooking into it, send me an email
or ping me on irc to make sure we don't duplicate the work)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?
Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.
On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.
anoncvs still serves the old cvs repository.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander <magnus@hagander.net> wrote:
On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.anoncvs still serves the old cvs repository.
Whens that due to be fixed? I imagine most of the buildfarm is testing
that still...
--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake
EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise Postgres Company
Peter Eisentraut wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.
heh - well nagios complained that the current tarballs are outdated, I
have not actually investigated further :)
But iirc snapshot generation is running against the main cvs repo and
not anoncvs.
Stefan
On Wed, Sep 22, 2010 at 12:19, Dave Page <dpage@pgadmin.org> wrote:
On Wed, Sep 22, 2010 at 11:11 AM, Magnus Hagander <magnus@hagander.net> wrote:
On Wed, Sep 22, 2010 at 11:29, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.anoncvs still serves the old cvs repository.
Whens that due to be fixed? I imagine most of the buildfarm is testing
that still...
Don't have an actual time for that. Figured we'd leave the old one
running until things have settled down.
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:
Hi all!
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to move
those to fetching from git?
I think this should be working now. I made this change to mk-snapshot,
and a similar one to mk-stable_snapshot. I have run the snapshot
generation for master, and the one for 8.4 should finish in a
minute or so. I have disabled the cronjobs until someone else has
verified the tarballs.
When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
We currently do back to 8.2 - should we keep that, or drop 8.2?
//Magnus
*** old/mk-snapshot Wed Sep 22 06:02:02 2010
--- mk-snapshot Wed Sep 22 07:09:13 2010
***************
*** 8,14 ****
export MAKE=gmake
export MD5SUM=md5! export CVSROOT=:ext:scrappy@cvs.postgresql.org:/cvsroot
export MAKEFLAGS=VERSION=snapshot
export TMPDIR=/usr/local/pgsql/snapshot
--- 8,14 ----
export MAKE=gmake
export MD5SUM=md5! export GIT_DIR=/usr/local/pgsql/git/postgresql/.git
export MAKEFLAGS=VERSION=snapshot
export TMPDIR=/usr/local/pgsql/snapshot
***************
*** 19,25 ****
then
rm -fr pgsql
fi
! /usr/bin/cvs -q export -rHEAD pgsql
cd pgsql
./configure
export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location)
--- 19,28 ----
then
rm -fr pgsql
fi
! mkdir pgsql
! git fetch
! git archive origin/master | tar xf - -C pgsql
!
cd pgsql
./configure
export OUTPUTFILE=$($MAKE -s $MAKEFLAGS distdir-location)--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 10:33, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:Hi all!
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to move
those to fetching from git?I think this should be working now. I made this change to mk-snapshot,
and a similar one to mk-stable_snapshot. I have run the snapshot
generation for master, and the one for 8.4 should finish in a
minute or so. I have disabled the cronjobs until someone else has
verified the tarballs.When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
We currently do back to 8.2 - should we keep that, or drop 8.2?
hmm I would say we should do all supported branches because those are
the ones that people might be interested in when looking for a fix on a
branch...
Stefan
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.
We need to have a fix for this before we can do any more backbranch
releases.
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.We need to have a fix for this before we can do any more backbranch
releases.
Fix for what, exactly? (assuming that people update their bf animals of course)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Peter Eisentraut wrote:
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.We need to have a fix for this before we can do any more backbranch
releases.
yeah having to touch ever BF member despite the fact that we have a
git->cvs gateway seems heavily annoying and not really expected. However
if this must be done oen should send a proper headsup and howto to the
buildfarm member list (people need to add the 9.0 branch as well so we
better tell them this now instead of having to do work twice).
Stefan
Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.We need to have a fix for this before we can do any more backbranch
releases.Fix for what, exactly? (assuming that people update their bf animals of course)
how exactly should people fix their BF members - is there any howto or
such on how to deal with the new git2cvs gateway?
Switching to git entirely is rather hard for some of my members for
example - at least not without investing hours of work...
Stefan
On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
We need to have a fix for this before we can do any more backbranch
releases.Fix for what, exactly? (assuming that people update their bf animals of course)
For the NLS workflow, we need to have some kind of CVS server that
serves up all supported branches. Changing the URL or the module name
or whatever is fine.
On Wed, Sep 22, 2010 at 13:28, Stefan Kaltenbrunner
<stefan@kaltenbrunner.cc> wrote:
Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 12:22 +0200, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.We need to have a fix for this before we can do any more backbranch
releases.Fix for what, exactly? (assuming that people update their bf animals of
course)how exactly should people fix their BF members - is there any howto or such
on how to deal with the new git2cvs gateway?
Andrew has posted info on how to convert them to git, I believe.
I don't think there are any for the git2cvs gateway, since it's not up yet.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
We need to have a fix for this before we can do any more backbranch
releases.Fix for what, exactly? (assuming that people update their bf animals of course)
For the NLS workflow, we need to have some kind of CVS server that
serves up all supported branches. Changing the URL or the module name
or whatever is fine.
I thought you had/were going to convert those to work natively with
git? I guess I misunderstood you.
Anyway, it's on my list to get done pretty quickly.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 13:45 +0200, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote:
For the NLS workflow, we need to have some kind of CVS server that
serves up all supported branches. Changing the URL or the module name
or whatever is fine.I thought you had/were going to convert those to work natively with
git? I guess I misunderstood you.
I think CVS should be eliminated from that process during the 9.1
release cycle, but we are looking for backbranch releases within the
next few weeks, and there is no time to get that done by then.
On 09/22/2010 06:22 AM, Magnus Hagander wrote:
Mind you, I think bf members will need tweaking anyway, since they
will assume the old style cvs layout. AFAICT, git-cvsserver will
export each branch as a module, rather than one module as pgsql. We
can obviously get that working for the HEAD branch (by just mapping it
to a branch called pgsql), but I don't know how doable that is for
backbranches.
Buildfarm owners should be moving aggressively towards using git. I'm
not aware of any platform it can't be used on.
I was planning on doing an audit around the end of the week and
following up with people who haven't migrated.
cheers
andrew
On Wed, Sep 22, 2010 at 13:45, Magnus Hagander <magnus@hagander.net> wrote:
On Wed, Sep 22, 2010 at 13:38, Peter Eisentraut <peter_e@gmx.net> wrote:
On ons, 2010-09-22 at 13:25 +0200, Magnus Hagander wrote:
On Wed, Sep 22, 2010 at 13:20, Peter Eisentraut <peter_e@gmx.net> wrote:
We need to have a fix for this before we can do any more backbranch
releases.Fix for what, exactly? (assuming that people update their bf animals of course)
For the NLS workflow, we need to have some kind of CVS server that
serves up all supported branches. Changing the URL or the module name
or whatever is fine.I thought you had/were going to convert those to work natively with
git? I guess I misunderstood you.Anyway, it's on my list to get done pretty quickly.
Started again on this.
Whenever I try to do something useful, it crashes with sqlite errors
that appear to be FreeBSD specific.
I'm going to try to upgrade all the ports on the box and give it
another try. Unfortunately, that tends to take a couple of hours - but
it'll get done eventually :-)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
Magnus Hagander wrote:
I think this should be working now. I made this change to mk-snapshot,
and a similar one to mk-stable_snapshot. I have run the snapshot
generation for master, and the one for 8.4 should finish in a
minute or so. I have disabled the cronjobs until someone else has
verified the tarballs.
Do you still need someone to do that, and what do you want done exactly?
When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
We currently do back to 8.2 - should we keep that, or drop 8.2?
hmm I would say we should do all supported branches because those are
the ones that people might be interested in when looking for a fix on a
branch...
FWIW, I think back to 8.2 is sufficient. The branches earlier than that
are going to be unsupported in a matter of weeks anyway. And if no one
has complained about the lack of nightly tarballs for them to date,
it's unlikely that they'll start complaining now.
regards, tom lane
On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> writes:
Magnus Hagander wrote:
I think this should be working now. I made this change to mk-snapshot,
and a similar one to mk-stable_snapshot. I have run the snapshot
generation for master, and the one for 8.4 should finish in a
minute or so. I have disabled the cronjobs until someone else has
verified the tarballs.Do you still need someone to do that, and what do you want done exactly?
Just a second set of eyes that the output looks reasonable for being a
snapshot generated now.
When I do, I'll also enable generation of snapshots of REL9_0_STABLE.
We currently do back to 8.2 - should we keep that, or drop 8.2?hmm I would say we should do all supported branches because those are
the ones that people might be interested in when looking for a fix on a
branch...FWIW, I think back to 8.2 is sufficient. The branches earlier than that
are going to be unsupported in a matter of weeks anyway. And if no one
has complained about the lack of nightly tarballs for them to date,
it's unlikely that they'll start complaining now.
Ok. 8.2 it is then - and I'll add 9.0
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Do you still need someone to do that, and what do you want done exactly?
Just a second set of eyes that the output looks reasonable for being a
snapshot generated now.
Sure, I can look. Where are these tarballs anyway?
regards, tom lane
On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Wed, Sep 22, 2010 at 16:50, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Do you still need someone to do that, and what do you want done exactly?
Just a second set of eyes that the output looks reasonable for being a
snapshot generated now.Sure, I can look. Where are these tarballs anyway?
They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
dev and stable/8.4 are updated.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Sure, I can look. �Where are these tarballs anyway?
They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
dev and stable/8.4 are updated.
Both of those look pretty sane from here.
regards, tom lane
On Wed, Sep 22, 2010 at 19:23, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Magnus Hagander <magnus@hagander.net> writes:
On Wed, Sep 22, 2010 at 17:08, Tom Lane <tgl@sss.pgh.pa.us> wrote:
Sure, I can look. Where are these tarballs anyway?
They're up on the ftp site - ftp://ftp.postgresql.org/pub/snapshot/.
dev and stable/8.4 are updated.Both of those look pretty sane from here.
Ok, I've re-enabled the cron job.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.
Developer docs are now building again.
On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote:
On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.Developer docs are now building again.
And NLS is also fixed.
On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut <peter_e@gmx.net> wrote:
On tor, 2010-09-23 at 11:07 +0300, Peter Eisentraut wrote:
On ons, 2010-09-22 at 12:29 +0300, Peter Eisentraut wrote:
On ons, 2010-09-22 at 10:33 +0200, Stefan Kaltenbrunner wrote:
It seems that the git move has broken the generation of the automatic
snapshot tarballs - has anybody yet looked into what it would take to
move those to fetching from git?Depends on what's "broken" about it, but I notice that the developer
docs and the NLS builds are also not updating. Perhaps something wrong
with the anoncvs service.Developer docs are now building again.
And NLS is also fixed.
Great. Thanks - that takes one more thing off the cvs requirement ;)
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes:
On Fri, Sep 24, 2010 at 07:15, Peter Eisentraut <peter_e@gmx.net> wrote:
And NLS is also fixed.
Great. Thanks - that takes one more thing off the cvs requirement ;)
Yeah. Maybe we don't need a cvsserver after all. Would it make more
sense to help Stefan get git running on his BSD boxes? If Bruce and
I could get it to work on our pet dinosaurs, I think it likely can
be gotten to work on spoonbill too.
regards, tom lane
On 09/24/2010 03:47 PM, Tom Lane wrote:
Yeah. Maybe we don't need a cvsserver after all. Would it make more
sense to help Stefan get git running on his BSD boxes? If Bruce and
I could get it to work on our pet dinosaurs, I think it likely can
be gotten to work on spoonbill too.
Yeah, I find it hard to believe that a machine can build postgres, with
openssl, but can't build git. The problem is that Stefan doesn't have
time to work on it, and I gather he's unable to give anyone else access.
cheers
andrew
On 09/24/2010 09:54 PM, Andrew Dunstan wrote:
On 09/24/2010 03:47 PM, Tom Lane wrote:
Yeah. Maybe we don't need a cvsserver after all. Would it make more
sense to help Stefan get git running on his BSD boxes? If Bruce and
I could get it to work on our pet dinosaurs, I think it likely can
be gotten to work on spoonbill too.Yeah, I find it hard to believe that a machine can build postgres, with
openssl, but can't build git. The problem is that Stefan doesn't have
time to work on it, and I gather he's unable to give anyone else access.
yeah don't worry too much, I will find a way to fix it will just take a
few more days until I get a bit of spare time...
Stefan