moving development branch activity to new git repo

On 21/09/10 18:28, Kevin Grittner wrote:

I just went to do my usual merge from the git version of HEAD (at
git://git.postgresql.org/git/postgresql.git), and it seemed to be
doing an awful lot of work to prepare to attempt the merge. That
leads me to think that the newly converted git, or a copy of it, is
now at that location, which is cool. But I have concerns about what
to do with my development branch off the old one.

I'm afraid that in spite of several attempts, I don't yet properly
have my head around the git approach, and fear that I'll muck things
up without a little direction; and I'd be surprised if I'm the only
one in this position.

Can someone give advice, preferably in the form of a "recipe", for
how to set up a new repo here based on the newly converted repo, and
merge the work from my branch (with all the related history) into a
branch off the new repo?

Some ideas:

A) Generate a patch in the old repo, and apply it to the new one.
Simple, but you lose the history.

B) git rebase. First "git fetch" the new upstream repository into your
local repository, and use git rebase to apply all the commits in your
private branch over the new upstream branch. You will likely get some
conflicts and will need to resolve them by hand, but if you're lucky
it's not a lot of work.

C) Git grafts. I just tested this method for our internal EDB
repository, and seems to work pretty well. You will need one line in
your .git/info/grafts file for each merge commit with upstream that you
have made. On each line you have 1. commitid of the merge commit 2.
commitid of the old PostgreSQL commit that was merged 3. commitid of the
corresponding PostgreSQL commit in the new repository. This lets you
continue working on your repository as you used to, merging and all, but
git diff will show that all the $PostgreSQL$ are different from the new
upstream repository.

I'd suggest that you just do A) and keep the old repository around for
reference.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On 09/21/2010 11:28 AM, Kevin Grittner wrote:

I just went to do my usual merge from the git version of HEAD (at
git://git.postgresql.org/git/postgresql.git), and it seemed to be
doing an awful lot of work to prepare to attempt the merge. That
leads me to think that the newly converted git, or a copy of it, is
now at that location, which is cool. But I have concerns about what
to do with my development branch off the old one.

I'm afraid that in spite of several attempts, I don't yet properly
have my head around the git approach, and fear that I'll muck things
up without a little direction; and I'd be surprised if I'm the only
one in this position.

Can someone give advice, preferably in the form of a "recipe", for
how to set up a new repo here based on the newly converted repo, and
merge the work from my branch (with all the related history) into a
branch off the new repo?

I was just mentioning to Magnus a couple of hours ago on chat that this
would create headaches for some people.

Basically, AIUI, you have to move the old repo aside and freshly clone
the new repo.

I haven't migrated my development trees yet, but I'm planning on simply
applying a diff from the old repo to a newly created branch in the new
repo. However, that does mean losing the private commit history. I'm not
sure much can be done about that, unless you migrate each commit
separately, which could be painful. Maybe some of the git gurus have
better ideas, though.

cheers

andrew

* Andrew Dunstan <andrew@dunslane.net> [100921 11:59]:

I was just mentioning to Magnus a couple of hours ago on chat that this
would create headaches for some people.

Basically, AIUI, you have to move the old repo aside and freshly clone
the new repo.

I haven't migrated my development trees yet, but I'm planning on simply
applying a diff from the old repo to a newly created branch in the new
repo. However, that does mean losing the private commit history. I'm not
sure much can be done about that, unless you migrate each commit
separately, which could be painful. Maybe some of the git gurus have
better ideas, though.

Someone mentioned "git rebase". That' probably going to be slow on
distint repositories too. The grafts mentioned will speed that up.

But probably the easiest way, if you have a nice clean history, is to
use "git formatpatch". This produces a nice "series" of patches, with
your commit message, and content, and dates, all preserved, ready for
re-applying (git am can do that automatically on the new branch), or
emailing, or whatever.

If you're history is a complicated tangle of merges because you
constantly just re-merge the "CVS HEAD" into your dev branch, then it
might be time to just do a massive "diff" and "apply" anyways ;-)

a.

--
Aidan Van Dyk Create like a god,
aidan@highrise.ca command like a king,
http://www.highrise.ca/ work like a slave.

Andrew Dunstan <andrew@dunslane.net> wrote:

Basically, AIUI, you have to move the old repo aside and freshly
clone the new repo.

I was assuming that, but it's good to have confirmation. What about
my repo at

http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git ?

Can that be reset to a copy of the new repo? (Or is that not really
beneficial?)

I haven't migrated my development trees yet, but I'm planning on
simply applying a diff from the old repo to a newly created branch
in the new repo. However, that does mean losing the private commit
history.

Yeah, I'd really rather not lose that.

I'm not sure much can be done about that, unless you migrate each
commit separately, which could be painful.

Perhaps. I might be able to use grep and sed to script it, though.
Right now I think I'd be alright to just pick off commits where the
committer was myself or Dan Ports. My bash-fu is tolerably good for
such purposes.

Maybe some of the git gurus have better ideas, though.

I'm all ears. ;-)

-Kevin

On September 21, 2010 12:08:49 pm Kevin Grittner wrote:

Andrew Dunstan <andrew@dunslane.net> wrote:

Basically, AIUI, you have to move the old repo aside and freshly
clone the new repo.

I was assuming that, but it's good to have confirmation. What about
my repo at

http://git.postgresql.org/gitweb?p=users/kgrittn/postgres.git ?

Can that be reset to a copy of the new repo? (Or is that not really
beneficial?)

I haven't migrated my development trees yet, but I'm planning on
simply applying a diff from the old repo to a newly created branch
in the new repo. However, that does mean losing the private commit
history.

Yeah, I'd really rather not lose that.

I'm not sure much can be done about that, unless you migrate each
commit separately, which could be painful.

Perhaps. I might be able to use grep and sed to script it, though.
Right now I think I'd be alright to just pick off commits where the
committer was myself or Dan Ports. My bash-fu is tolerably good for
such purposes.

Maybe some of the git gurus have better ideas, though.

I'm all ears. ;-)

-Kevin

Here's a quick and easy way to move dev history to a new repo:

$ cd postgresql.old
$ git checkout yourbranch

# stream your commits into a "patch mailbox"
$ git format-patch --stdout master..HEAD > patches.mbox

# switch to the new repo
$ cd ../postgresql

# create a branch if not already
$ git checkout -b yourbranch

# apply the "patch mailbox"
$ git am ../postgresql.old/patches.mbox

That should do the trick. Your dev history will be kept.

Elvis

On 09/21/2010 12:07 PM, Aidan Van Dyk wrote:

But probably the easiest way, if you have a nice clean history, is to
use "git formatpatch". This produces a nice "series" of patches, with
your commit message, and content, and dates, all preserved, ready for
re-applying (git am can do that automatically on the new branch), or
emailing, or whatever.

Ah. I thought there was something like this but for some reason when I
went looking for it just now I failed to find it.

Thanks for the info. This looks like the best way to go.

cheers

andrew

At 2010-09-21 11:59:09 -0400, andrew@dunslane.net wrote:

However, that does mean losing the private commit history. I'm not
sure much can be done about that, unless you migrate each commit
separately, which could be painful.

It doesn't have to be painful.

Determine what patches from the old repository you want to apply, and
create a branch in the newly-cloned repository to apply them to. Then
use (cd ../oldrepo;git format-patch -k --stdout R1..R2)|git am -3 -k"
to apply a series of patches (between revisions R1 and R2; adjust as
needed) to your branch (i.e. when you have it checked out).

See git-format-patch(1) and git-am(1) for more details (or feel free
to ask if you need more help).

-- ams

Elvis Pranskevichus <el@prans.net> wrote:

Here's a quick and easy way to move dev history to a new repo:

$ cd postgresql.old
$ git checkout yourbranch

# stream your commits into a "patch mailbox"
$ git format-patch --stdout master..HEAD > patches.mbox

# switch to the new repo
$ cd ../postgresql

# create a branch if not already
$ git checkout -b yourbranch

# apply the "patch mailbox"
$ git am ../postgresql.old/patches.mbox

That should do the trick. Your dev history will be kept.

Thanks for the recipe. (And thanks to all others who responded.)

That still leaves me wondering how I get that out to my public git
repo without someone resetting it on the server. Or do I have the
ability to clean out the old stuff at:

ssh://git@git.postgresql.org/users/kgrittn/postgres.git

so that I can push the result of the above to it cleanly?

-Kevin

On Tue, Sep 21, 2010 at 20:01, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

Elvis Pranskevichus <el@prans.net> wrote:

Here's a quick and easy way to move dev history to a new repo:

$ cd postgresql.old
$ git checkout yourbranch

# stream your commits into a "patch mailbox"
$ git format-patch --stdout master..HEAD > patches.mbox

# switch to the new repo
$ cd ../postgresql

# create a branch if not already
$ git checkout -b yourbranch

# apply the "patch mailbox"
$ git am ../postgresql.old/patches.mbox

That should do the trick.  Your dev history will be kept.

Thanks for the recipe.  (And thanks to all others who responded.)

That still leaves me wondering how I get that out to my public git
repo without someone resetting it on the server.  Or do I have the
ability to clean out the old stuff at:

ssh://git@git.postgresql.org/users/kgrittn/postgres.git

so that I can push the result of the above to it cleanly?

a git push *should* work, but we've seen issues with that.

The cleanest is probably if I wipe the repo on git.postgresql.org for
you, and you then re-push from scratch. Does thta work for you?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 21/09/10 21:01, Kevin Grittner wrote:

That still leaves me wondering how I get that out to my public git
repo without someone resetting it on the server. Or do I have the
ability to clean out the old stuff at:

ssh://git@git.postgresql.org/users/kgrittn/postgres.git

so that I can push the result of the above to it cleanly?

git push --force allows you to push the new branches over the old ones.

I don't think it will automatically garbage collect the old stuff
though, so the repository will be bloated until "git gc" runs (with
--aggressive or something, not sure). I don't know when or how that
happens in the public repos.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Magnus Hagander <magnus@hagander.net> wrote:

The cleanest is probably if I wipe the repo on git.postgresql.org
for you, and you then re-push from scratch. Does thta work for
you?

Sure. Thanks.

-Kevin

On Tue, Sep 21, 2010 at 20:16, Kevin Grittner
<Kevin.Grittner@wicourts.gov> wrote:

Magnus Hagander <magnus@hagander.net> wrote:

The cleanest is probably if I wipe the repo on git.postgresql.org
for you, and you then re-push from scratch. Does thta work for
you?

Sure.  Thanks.

done, should be available for push now.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On tis, 2010-09-21 at 20:04 +0200, Magnus Hagander wrote:

The cleanest is probably if I wipe the repo on git.postgresql.org for
you, and you then re-push from scratch.

We probably need a solution that doesn't require manual intervention for
everyone separately.

On Tue, Sep 21, 2010 at 20:28, Peter Eisentraut <peter_e@gmx.net> wrote:

On tis, 2010-09-21 at 20:04 +0200, Magnus Hagander wrote:

The cleanest is probably if I wipe the repo on git.postgresql.org for
you, and you then re-push from scratch.

We probably need a solution that doesn't require manual intervention for
everyone separately.

Are there really that many? If nothing else, it's a good way to figure
out which repos are actually used ;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Elvis Pranskevichus <el@prans.net> wrote:

# apply the "patch mailbox"
$ git am ../postgresql.old/patches.mbox

That's not working for me.

Applying: Provide two macros for categorizing current transaction
isolation level (IsXactIsoLevelXactSnapshotBased and
IsXactIsoLevelFullySerializable) to replace the
IsXactIsoLevelSerializable macro. Adjust comments to reflect the
distinction, and rename a now-misleading variable.
error: patch failed: src/backend/catalog/index.c:2133
error: src/backend/catalog/index.c: patch does not apply
error: patch failed: src/backend/commands/trigger.c:2319
error: src/backend/commands/trigger.c: patch does not apply
error: patch failed: src/backend/executor/execMain.c:1538
error: src/backend/executor/execMain.c: patch does not apply
error: patch failed: src/backend/executor/nodeLockRows.c:130
error: src/backend/executor/nodeLockRows.c: patch does not apply
error: patch failed: src/backend/executor/nodeModifyTable.c:326
error: src/backend/executor/nodeModifyTable.c: patch does not apply
error: patch failed: src/backend/utils/adt/ri_triggers.c:3314
error: src/backend/utils/adt/ri_triggers.c: patch does not apply
error: patch failed: src/backend/utils/time/snapmgr.c:37
error: src/backend/utils/time/snapmgr.c: patch does not apply
error: patch failed: src/include/access/xact.h:32
error: src/include/access/xact.h: patch does not apply
Patch failed at 0001 Provide two macros for categorizing current
transaction isolation level (IsXactIsoLevelXactSnapshotBased and
IsXactIsoLevelFullySerializable) to replace the
IsXactIsoLevelSerializable macro. Adjust comments to reflect the
distinction, and rename a now-misleading variable.
When you have resolved this problem run "git am --resolved".
If you would prefer to skip this patch, instead run "git am --skip".
To restore the original branch and stop patching run "git am
--abort".

For the record, this branch was regularly merged with changes from
the master branch of the old PostgreSQL git repo which was a copy of
CVS head. I filtered out the 167 non-merge commits on my
serializable branch since 8 Jan 2010. Is there any practical way to
resolve this so that I can keep the history?

-Kevin

On September 21, 2010 07:32:57 pm Kevin Grittner wrote:

Elvis Pranskevichus <el@prans.net> wrote:

# apply the "patch mailbox"
$ git am ../postgresql.old/patches.mbox

That's not working for me.

Applying: Provide two macros for categorizing current transaction
isolation level (IsXactIsoLevelXactSnapshotBased and
IsXactIsoLevelFullySerializable) to replace the
IsXactIsoLevelSerializable macro. Adjust comments to reflect the
distinction, and rename a now-misleading variable.
error: patch failed: src/backend/catalog/index.c:2133
error: src/backend/catalog/index.c: patch does not apply
error: patch failed: src/backend/commands/trigger.c:2319
error: src/backend/commands/trigger.c: patch does not apply
error: patch failed: src/backend/executor/execMain.c:1538
error: src/backend/executor/execMain.c: patch does not apply
error: patch failed: src/backend/executor/nodeLockRows.c:130
error: src/backend/executor/nodeLockRows.c: patch does not apply
error: patch failed: src/backend/executor/nodeModifyTable.c:326
error: src/backend/executor/nodeModifyTable.c: patch does not apply
error: patch failed: src/backend/utils/adt/ri_triggers.c:3314
error: src/backend/utils/adt/ri_triggers.c: patch does not apply
error: patch failed: src/backend/utils/time/snapmgr.c:37
error: src/backend/utils/time/snapmgr.c: patch does not apply
error: patch failed: src/include/access/xact.h:32
error: src/include/access/xact.h: patch does not apply
Patch failed at 0001 Provide two macros for categorizing current
transaction isolation level (IsXactIsoLevelXactSnapshotBased and
IsXactIsoLevelFullySerializable) to replace the
IsXactIsoLevelSerializable macro. Adjust comments to reflect the
distinction, and rename a now-misleading variable.
When you have resolved this problem run "git am --resolved".
If you would prefer to skip this patch, instead run "git am --skip".
To restore the original branch and stop patching run "git am
--abort".

For the record, this branch was regularly merged with changes from
the master branch of the old PostgreSQL git repo which was a copy of
CVS head. I filtered out the 167 non-merge commits on my
serializable branch since 8 Jan 2010. Is there any practical way to
resolve this so that I can keep the history?

-Kevin

Instead of filtering non-merge commits I would suggest doing git rebase on the
latest revision of the old git repo. That way you will get a set of commits
that should apply cleanly. The reason it is failing now is that it is
impossible for git am to do a 3-way merge without the original git objects,
which are obviously not available in the new repo.

Elvis

Elvis Pranskevichus <el@prans.net> wrote:

Instead of filtering non-merge commits I would suggest doing git
rebase on the latest revision of the old git repo. That way you
will get a set of commits that should apply cleanly. The reason
it is failing now is that it is impossible for git am to do a
3-way merge without the original git objects, which are obviously
not available in the new repo.

Well, that didn't work much better. It applied, but it started in
June, and the "big" file, which has been in constant flux since
January suddenly springs into existence in July. :-( The last few
commits look right, but it gets pretty trashy pretty quickly before
that.

What *did* work is to take a fresh of the new repo, branch it as of
the point in time that I created my original branch, and take the
first mbox entry of the 167, which starts like this:

From 07ea78aaafe22cbbb0ffdedbfcf78404abbdbc70 Mon Sep 17 00:00:00

2001
From: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Date: Fri, 8 Jan 2010 15:39:12 -0600

and change the first line to point to the point at which this was
applied:

From 369494e41fe408f103884032f477555ba134a0a8 Fri Jan 8 09:06:05

2010

That applies fine. It's an encouraging start, but I'm not clear on
exactly what I have to do to get the rest of these commits merged in
with commits from the master branch cleanly. Some fix-up work is OK
with me. Do I need to look at the old log and manually interleave
merges to the branch and manual commits in the original pattern to
make such a scheme work?

-Kevin

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> wrote:

Well, that didn't work much better.

I decided I'd reached my limit on this. I fell back to a suggestion
made a while back, and just created a patch from the old repository
and applied it to the new one. I've pushed it to the public repo,
but haven't yet had a chance to confirm that all is well. I will
keep the old repo in case there is any need to prove the development
history. (Unlikely, but it seems prudent to cover the bases.)

So, no further advice on this topic needed here.

-Kevin