debian tips/ tips/ Hacking on an open source project using bzr

I have recently been providing patches to a couple of open source projects that use svn to keep their work versioned.

The difficulty for me is that I will create a patch, and there is no guarantee it will be accepted. However I want to create multiple patches and be able to provide them separately.

A good way to solve this is by using branches. They allow changes to be developed in parallel, without affecting one another. The problem is that with svn you need commit access to the repository to be able to branch easily.

One workaround would be to have several checkouts, and create a patch in each, but this is quite expensive, requiring a full checkout for each branch. This takes up both disk space and network bandwidth.

I have recently started using bzr to manage my work, and one of it's features is that branching is cheap and can be done locally. So I would like to use bzr to manage my work in svn.

To do this I used the following method.

      # create a directory to hold all of the work
      mkdir project
      cd project
      # Create a bzr repository (reduces disk usage later)
      bzr init-repo --trees .
      # Get the upstream source
      svn co http://wherever/some/project upstream

This creates a directory containing the upstream source. The idea is to leave this as untouched as possible. Try not to make any changes to this code, and don't build from there, as this might confuse you later.

Now create some bzr branches to manage your work

      # Create a bzr branch for the upstream code.
      cd upstream
      bzr init
      bzr add .
      bzr ci -m'Import of project'
      # Create the master branch
      cd ../
      bzr branch upstream main

In the main branch you should make any local modifications that are necessary for your system, but that wont be sent upstream.

      cd main
      # make local changes
      bzr ci -m'Add local changes'

Now whenever you want to work on a patch (say add a feature), create a new branch for it. This will keep the changes independent.

      # Create the working branch
      cd ..
      bzr branch main feature
      cd feature
      # Implement the feature
      bzr ci -m'Added feature'

Now you should make sure the code is up to date with upstream to provide a clean patch (you can do this often to reduce the amount of conflict resolution that you have to do)

      cd ../upstream
      svn update

This will pull upstream's changes. There should not be any conflicts as you didn't change any files. You must watch out for any added files though (A lines of the update output). bzr unknowns can help you here. If there were any added files you must add them to bzr.

      bzr add file
      # or bzr add $(bzr unknowns)
      bzr ci -m'Merge upstream svn revision 1234'

This means that your upstream branch is up to date. You now must propogate these to your feature branch. You should go through your main branch first though to maintain your local changes.

      cd ../main
      # Pull in the upstream changes
      bzr merge
      # If you have any conflicts then edit the files and
      bzr resolved file
      # When all is well (bzr status and bzr diff can tell 
      # you what is going on)
      bzr ci -m'Merge upstream svn revision 1234'

You can now do the same for your feature branch.

      cd ../feature
      bzr merge
      # bzr resolved if needed
      bzr ci -m'Merge upstream svn revision 1234'

This might seem like a lot of work, but it can be done quickly, especially if development is quiet. However I find the benifits of this to outweigh this work. It is also said the bzr has superior merging ability to svn, so perhaps doing this will reduce the amount of manual conflict resolution you will have to do.

So now you have a feature implemented in a branch and you want to send a patch upstream. This is very easy. Due to the separation of feature branches you will always have clean diffs, and branching from a bzr branch containing your local modifications means that your diffs will not contain your changes. So to create these diffs do

      bzr diff -r branch:../main > ../feature.diff

You can then review the diff and send it upstream.

Now you can leave your feature branch alone, and create another branch if you want to do more work. If upstream asks for more work, you can just go back to the branch and make the necessary changes, and then send another diff.

Now, if the patch gets accepted then your changes will come down in the next update from svn, and then will propogate to all of your branches. You will not get any conflicts, even if upstream makes some changes to your code.

If your patch is not accepted, but you would like to keep it locally, you can do this easily. Just merge your branch back in to your main branch, which will then propogate to all of your branches.

      cd ../main
      bzr merge ../feature
      # If upstream partially included your patch you may have to
      # deal with conflicts here.
      bzr ci -m'Add feature'

Now when you merge from any of your feature branches they will get these changes, and so their diffs will not contain it, keeping the patches separate.

If you want to split up a feature implementation, so that you can work with upstream to get it included in parts you can create a feature branch that depends on another one.

     bzr branch main small.change
     bzr branch small.change large.change

Now you propogate changes to large.change through small.change. You can then generate 3 patches.

     # Create a diff to implement small.change
     cd small.change
     bzr diff -r branch:../main
     # Create a diff to implement large.change
     cd large.change
     bzr diff -r branch:../main
     # Create a diff to implement large.change that is dependent on 
     # small.change. Useful if you would like to discuss the second
     # part of the feature on it's own.
     cd large.change
     bzr diff -r branch:../small.change

So you can see here how bzr can give some flexibilty when hacking on a project, no matter what vcs they use.