git

Revision 9 as of 2018-06-13 16:12:15

Clear message

The DesktopTeam uses git to manage its packages. Here's how to do some common things. You need git-buildpackage installed for most of this.

Suggested configuration is available in the end. If you are using it, any local changes to the git repor

General concepts

Branches and repo layout

We will use a layout with 3 remote repositories:

$ git remote show
origin          # will point to launchpad (default for pull and push without any argument)
debian          # will point to debian salsa repository, to cherry-pick easily fixes from debian and rebase
upstream        # will point to upstream repository, to cherry-pick easily upstream fixes

Launchpad remote branches

Let's look at the various branches in the origin (launchpad) remote repository:

$ git remote show origin 
* remote origin
[…]
  HEAD branch: ubuntu/master
  Local branches configured for 'git pull':
    pristine-tar  merges with remote pristine-tar
    ubuntu/bionic merges with remote ubuntu/bionic
    ubuntu/master merges with remote ubuntu/master
  Local refs configured for 'git push':
    pristine-tar    pushes to pristine-tar    (up to date)
    ubuntu/bionic   pushes to ubuntu/bionic   (up to date)
    ubuntu/master   pushes to ubuntu/master   (up to date)
    upstream/latest pushes to upstream/latest (up to date)
  • ubuntu/master branch is the content of the latest development release. Work mostly happens here. It's the default branch.

  • ubuntu/bionic branch is the content for the corresponding release, once a particular release it out.

  • pristine-tar is an internal gbp-buildpackage branch, used to reconstruct release tarballs (using upstream/latest). You are not intended to interact with itdon't touch directly, but via gbp-buildpackage tools.

  • upstream/latest branch corresponds to the latest master branch from the upstream repository. It can be used to cherry-pick works. Updating it is manual. It's needed to help reconstructing the release tarballs.

We find back those 4 branches locally:

  • ubuntu/master -> pull and push from origin (launchpad) remote repository, on ubuntu/master branch (origin/ubuntu/master)

  • ubuntu/bionic -> pull and push from origin (launchpad) remote repository, on ubuntu/bionic branch (origin/ubuntu/bionic)

  • pristine-tar -> pull and push from origin (launchpad) remote repository, on pristine-tar branch (origin/pristine-tar)

  • upstream/latest -> pull from upstream remote repository, on the master branch (upstream/master), and push to origin (launchpad) remote repository, on upstream/latest branch (origin/upstream/master).

Debian remote branches

In addition, we'll have at least another branch tracking debian remote repository. Their master branch is named debian/master:

  • debian/master -> pull (no push, unless you are a debian developer) from debian remote repository, on debian/master branch. This will thus be "debian/debian/master". Don't be mislead by the fact that branch names have "/" in them, and so in remote/branch schema, we end up with debian/debian/master for the remote branch name.

Upstream remote branches

We have upstream/master tracking the upstream repo on master branch. We can track any additional branches as needed for cherry-picking fixes.

For instances:

$ git remote show upstream
* remote usptream
[…]
  HEAD branch: master
[…]
    gnome-3-28                                  tracked
[…]
  Local branch configured for 'git pull':
    upstream/latest merges with remote master

Let's checkout locally gnome-3-28 branch and make it the current checkout:

$ git checkout -b upstream/gnome-3-28 upstream/gnome-3-28
Branch 'upstream/gnome-3-28' set up to track remote branch 'gnome-3-28' from 'upstream'.
Switched to a new branch 'upstream/gnome-3-28'

We asked to create a new local branch named upstream/gnome-3.28 (first argument), tracking the remote upstream, on the branch gnome-3-28 (second argument).

Local branches

To sum up all this, for instance ([] are tracked branch):

$ git branch -vv
  debian/master   c02b29966 [debian/debian/master] Revert "Don't recommend libnss-myhostname since it doesn't seem needed any more (LP: #1766575)"
  pristine-tar    e59ab7422 [origin/pristine-tar] pristine-tar data for gnome-control-center_3.28.1.orig.tar.xz
  ubuntu/bionic   59adb2d88 [origin/ubuntu/bionic] Reintegrate 1:3.28.1-0ubuntu1.18.04.1
* ubuntu/master   63337a4cd [origin/ubuntu/master] releasing package gnome-control-center version 1:3.28.1-0ubuntu5
  upstream/latest 0bd90ca34 [upstream/master] region: Show scrollbars if needed

Note that the fact we push upstream/lastest to origin/usptream/latest is the only missing information in that command.

With the 3-28 branch local checkout, we have in addition:

  upstream/gnome-3-28 4f3035eff [remotes/upstream/gnome-3-28] user-accounts: Force symlink creation

Note that to disambiguate local branch and remote + branch name being the same, git prints the remote branch as remotes/upstream/gnome-3-28 instead of upstream/gnome-3-28. Both are equivalents when using a remote argument.

Local changes

With the suggested configuration (see it at the end of the page), any non committed changes (file modifications, additions or removal) will halt the build, as they won't be included as a safety reminder.

You can either:

  • ignore them, appending --git-ignore-new. The resulting package and build won't include those uncommitted changes. (../build-area/<your-package> will match the last commit).

  • force using the current directory with your local modifications instead of build-area, and include (despite the "ignore-new" option) any local modifications to the package. For this, append --git-ignore-new --git-export-dir=""

check out a package

Launchpad remote

If the Vcs-Git field is correct, you can

$ debcheckout --git-track=* glib2.0
declared git repository at https://git.launchpad.net/~ubuntu-desktop/ubuntu/+source/glib2.0
git clone https://git.launchpad.net/~ubuntu-desktop/ubuntu/+source/glib2.0 glib2.0 ...
[…}

Otherwise, you can check out any git-buildpackage repository manually

$ gbp clone lp:~ubuntu-desktop/ubuntu/+source/gnome-control-center
gbp:info: Cloning from 'lp:~ubuntu-desktop/ubuntu/+source/gnome-control-center'

$ cd gnome-control-center
$ git branch
  pristine-tar
* ubuntu/master
  upstream/latest

We want to push tags automatically, as gbp relies on them (this is a local configuration):

$ git config push.followTags true

Add upstream remote and configure tracking

$ git remote add -f upstream git@gitlab.gnome.org:GNOME/gnome-control-center.git
$ git branch -u upstream/master upstream/latest 
$ git config branch.upstream/latest.pushremote origin

Add debian remote

$ git remote add -f debian https://salsa.debian.org/gnome-team/gnome-control-center.git
$ git branch debian/master debian/debian/master 

You should now have 4 branches correctly setup and tracking the correct branch with expected configuration.

Day to day operation

Building a binary or source package

  • Binary package:

$ gbp buildpackage -S
[…]
  • Source package:

$ gbp buildpackage -S
[…]

With the proposed configuration, the artefacts will all end up in ../build-area (including tarball, which is reconstructed from the pristine-tar + upstream/latest branch) and build directory is cleaned up.

Useful tips

Some potential cases:

  • Don't purge the **build-area** directory after building: gbp buildpackage --git-no-purge

  • Build current branch with local uncommitted modifications: gbp buildpackage --git-ignore-new --git-export-dir=""

Working on patches

  1. Ensure you are on the correct ubuntu/ branch (or your local experimental-feature branch).

$ git checkout ubuntu/master # for instance
  1. Turn each patches in debian/patches as commits.

$ gbp pq rebase
gbp:info: Switching to 'patch-queue/ubuntu/master'
Current branch patch-queue/ubuntu/master is up to date.

-> Now, we have every patch referenced in debian/patches/series applied as separate commits on top of ubuntu/master, in a patch-queue/ubuntu/master branch, and we switched to that current checkout.

For instance

$ git missing ubuntu/master.. # git log would also show the commits
* 11a3a8cc6 - (HEAD -> patch-queue/ubuntu/master) [PATCH] night-ligth-dialog: Avoid dereferencing invalid p
ointer (5 minutes ago)
* da06252e1 - [PATCH 4/4] thunderbolt: move to the 'Devices' page (5 minutes ago)
* 2c9f5bcbb - [PATCH 3/4] thunderbolt: new panel for device management (5 minutes ago)
* 660e9e633 - [PATCH 2/4] shell: Icon name helper returns symbolic name (5 minutes ago)
* a78ae89dd - [PATCH 1/4] shell: Don't set per-panel icon (5 minutes ago)
[…]

3. Hack… We can git add/git commit. Any new commit will be a separate patch applied at the end of debian/patches/series. Commit description will be then converted to the patch description.

Remember previous tip of gbp buildpackage --git-ignore-new --git-export-dir="" to build a package with current content without having to switch branch.

4. optional: Reorder patches If we don't want this patch to be the last one, we can use an interactive git rebase:

$ git rebase -i ubuntu/master

That way, we only see the patches commit, can reorder them, amend or stash them. Removing one will also remove the patch from debian/patches/series

5. reapply all changes to the original branch:

$ gbp pq export
gbp:info: On 'patch-queue/ubuntu/master', switching to 'ubuntu/master'
gbp:info: Generating patches from git (ubuntu/master..patch-queue/ubuntu/master)

6. modify changelog, git add, git commit

Note that gbp pq will unfuzz a lot of patches everytime you use it, creating some noise. Please add those changes separated from your new or modified patch hack.

Pick some upstream commits

  1. Refresh upstream repo if needed:

$ git checkout upstream/latest
$ git pull
$ git push # to update the origin upstream/latest repo for others
  1. Cherry-pick one commit as a patch (here on ubuntu/master branch):

$ git checkout ubuntu/master
$ gbp pq rebase
$ git log -p upstream/latest
# get a hash commit you want to cherry-pick
$ git cherry-pick <hash>
$ gpq pq export
# update changelog
$ git add debian/patches/* debian/changelog
$ git commit

See previous section for more info on gbp pq.

Checkout and track an existing branch for an older ubuntu release

$ git checkout ubuntu/bionic

You are on the ubuntu/bionic branch tracking origin/ubuntu/bionic. git pull and push will sync up that branch with remote launchpad repository

Checkout and track an existing branch for a given upstream release

We already presented that, the difference with above is that we aren't tracking the "origin" repository (default), but the upstream one. The syntax is slightely different:

$ git checkout -b upstream/gnome-3-28 upstream/gnome-3-28

The first part is the local branch name "upstream/gnome-3-28" tracking the **gnome-3-28** branch from **upstream** repository (second part). We ask to create the branch "-b".

Then, once local branch is created, for future switch, it's like any other branch:

$ git checkout ubuntu/master
$ git checkout pristine-tar
$ git checkout upstream/gnome-3-28

Merge a new upstream version

On the branch you want to have version imported (like ubuntu/bionic here)

$ git checkout ubuntu/bionic
$ gbp import-orig ../gnome-control-center-3.28.2.tar.xz # can be as well --uscan to let it download and import
What is the upstream version? [3.28.2] 
gbp:info: Importing '../gnome-control-center-3.28.2.tar.xz' to branch 'upstream/latest'...
gbp:info: Source package is gnome-control-center
gbp:info: Upstream version is 3.28.2
gbp:info: Replacing upstream source on 'ubuntu/bionic'
gbp:info: Successfully imported version 3.28.2 of ../gnome-control-center-3.28.2.tar.xz

Release a package

# on corresponding branch…
$ dch -r ""
$ debcommit -r
$ gbp buildpackage -S
$ dput …
$ git push

This will push all tracked branches to launchpad if you made any change: ubuntu/master, ubuntu/old-series, pristine-tar, upstream/lastest. Tags will also be pushed.

Configurations

~/.gitconfig

To easily access to launchpad git repositories with git clone lp:~name-or-team/repo, you should add to your .gitconfig file the following paragraph:

[url "git+ssh://<your_user_name>@git.launchpad.net/"]
        insteadof = lp:

See https://help.launchpad.net/Code/Git for more information on git configuration with launchpad.

~/.gbp.conf

git-buildpackage uses this file for its default configuration; some tweaks are recommended.

[buildpackage]
# use this for exporting first your directory in a ../build-area:
export-dir = ../build-area/
# automatically GPG sign tags with associated keyid
sign-tags = True
keyid = 0xGPGKEYID

See gbp.conf(5) and man of gbp-buildpackage for more options.

merge with debian

convert a package to git for the first time

XXX: flesh this out

  • clone debian's repository
  • git remote add gnome <url to gnome git repository>; git fetch gnome

  • git remote add lp <url to lp>

  • find the place we diverged from Debian, and make an ubuntu/master branch at that tag - for example if we're on 3.28.1-2ubuntu9, branch from debian/3.28.1-2; if we're on 3.28.2-0ubuntu1, go back in history until you find a non-0 version.

  • on ubuntu/master, run gbp import-dsc --debian-branch=ubuntu/master --debian-tag='ubuntu/%(version)s' ../path/to/the/current/dsc

  • Fix up debian/control.in - mention the Launchpad URLs in Vcs-* and copy Debian's to XS-Debian-Vcs-*

  • Modify or create gbp.conf containing:

[DEFAULT]
pristine-tar = True
debian-branch=ubuntu/master
upstream-branch=upstream/latest
upstream-vcs-tag = %(version)s
debian-tag = ubuntu/%(version)s

note for a stable release that'll be something like

[DEFAULT]
pristine-tar = True
debian-branch=ubuntu/bionic
upstream-branch=upstream/3.28.x
upstream-vcs-tag = %(version)s
debian-tag = ubuntu/%(version)s
  • Add a changelog entry for the above and commit it
  • Now you can work with the repository
  • When finished, push the ubuntu/ branch and tags, any relevant upstream/ branches and tags and the pristine-tar branch. Visit the repository page on Launchpad, and set the default branch to ubuntu/master.