Updating rustc and cargo in ubuntu

Scope

We maintain rustc and cargo in ubuntu almost entirely to support building firefox. If you're a rust developer looking to work on your own rust programs, you are probably going to have a better experience (for now, at least) using rustup.

Generalities

The maintenance uses git-ubuntu and the git merge workflow both for merging upstream updates from debian and maintaining the stacks of packages required to backport rustc and cargo back to supported series.

PPAs and pockets and timelines

The ultimate destination for rustc and cargo is the security pocket of all supported releases. They are only copied there though when the version of firefox that requires them is copied there. All PPAs mentioned here must only depend on the security pocket, not updates.

The idealised timeline goes something like this (for supported series, for devel we just upload rustc and cargo like any other packages):

1. upstream firefox master changes to depend on a new version of rustc (and cargo)
2. these versions of rustc and cargo are uploaded to ppa:ubuntu-mozilla-security/rust-updates. I (mwhudson) sometimes use yet another PPA (ppa:mwhudson/rust-stuff) for my initial uploads.
3. when the version of firefox with the new rustc version requirement hits beta, rustc and cargo are binary copied to ppa:ubuntu-mozilla-security/rust-next. I think pre-release versions of firefox are built in some PPA I don't know about that depends on this one.
4. when this version of firefox is released, rustc and cargo and binary copied to ppa:ubuntu-mozilla-security/ppa and firefox is built in this PPA.
5. when firefox has built, firefox and rustc and cargo are all copied to the security pocket.

It doesn't always happen like this of course. In particular, upstream doesn't always leave long enough between master switching to a new version and beta for us to get rust-updates updated before beta.

Updating for the devel series

As above, this uses the git ubuntu merge workflow. I have a repo that has remotes for the package import branch on launchpad, the Debian maintainers's git repo from salsa and lp:~canonical-foundations/ubuntu/+source/rustc .

To merge a new version, 1.$X, I do something like this.

1. first note the tag of the version from debian you want to merge, this can be the Debian maintainer's tag (e.g. debian/1.32.0+dfsg1-3) or the package importers tag (e.g. pkg/import/1.30.0+dfsg1-1_exp2 -- sometimes you end up merging from experimental). Let's call this $newdebtag.

2. $ git checkout -b merge-1.$X merge-1.$((X-1))

3. $ git rebase -i $newdebtag --keep-empty.

   1. You might want to reword the first, empty, commit to indicate where the merge is from this time (unstable vs experimental vs NEW...)
4. remove the bottom five or so commits: merge-changelogs, reconstruct-changelog, ubuntu-meta, updating upstream source and "releasing package rust version ..."
5. work through the rebase, fixing conflicts as you go

6. Note the git tag for the last release we merged from debian, let's call this $olddebtag (I think this can be either from Debian or from the package importer, but maybe it needs to match $debtag in that. To be honest I often flail a bit at this step and just try things until it works)

7. $ git-ubuntu.merge-changelogs $olddebtag merge-1.$X $newdebtag

8. $ git commit -am 'merge-changelogs'

9. $ git reconstruct-changelog $newdebtag

10. Edit the version number in debian/changelog, the default is probably wrong and the +dfsg1 in the upstream version number needs to be changed to +dfsg1+llvm.

11. $ git commit -am 'reconstruct-changelog'

12. $ update-maintainer

13. $ git commit -am 'ubuntu-meta'

14. Import a version of the orig that bundles llvm: $ uscan. Now this has a couple of pitfalls:

    1. The tarball is enormous and will make mk-origtargz from older releases choke. Install devscripts from disco or better (and even then it still takes quite a long time).

    2. The rust download page only lists the current release. If we're a bit behind and are packaging an old release (and we can't skip releases) you have to grab the source tarball directly. Luckily the URL of the old source tarball is very predictable. Then you have to run mk-origtargz by hand, which is the super memorable "mk-origtargz --package rustc --version 1.33.0 --repack-suffix +dfsg1+llvm --signature 1 --signature-file ../rustc-1.33.0-src.tar.xz.asc --repack --compression xz --directory .. --copyright-file debian/copyright ../rustc-1.33.0-src.tar.xz" (with changes as needed for a different version).

15. Import the freshly made orig tarball. I have a fairly hair raising script I use for this but that's probably massive overkill.

16. Check that this update only touches src/llvm. If it doesn't something odd is going on, maybe you didn't download the orig that matches the version from debian.

17. At this stage I usually edit debian/changelog to append ~ppa1 to the version number, replace UNRELEASED with the name of the devel series and upload to ppa:mwhudson/rust-stuff.

18. If that builds OK, UNRELEASED->devel in changelog, $ debcommit -r -a --sign-tag and upload to devel! Don't forget to push the new commits and tags to Launchpad.

Backports

This process has fewer steps than the devel update but can be trickier (I guess because devel and Debian unstable are usually more or less in sync).

For reasons that are hopefully obvious, work your way from newer to older series.

To backport version 1.$X to an older series, $SERIES, I do:

1. git checkout -b $SERIES-1.$X $SERIES-1.$((X-1))

2. git rebase -i --keep-empty merge-1.$X

   1. stop ("e") on each commit that adds patches to check that the patches apply, and fix those up before continuing.

3. dch -b -v $(dpkg-parsechangelog)~$(ubuntu-distro-info --series=$SERIES --release | awk '{ print $1 }').1 "" or something similar

4. git reconstruct-changelog merge-1.$X

5. git commit -am reconstruct-changelog

6. At this point I usually do a test build in the rust-stuff PPA.

7. Once ready to uploaded, dch -D $SERIES -r ''

8. debcommit -a -r --sign-tag

9. Upload to ppa:ubuntu-mozilla-security/rust-updates.

An extremely valuable tip: if you have to make changes to a patch added by a commit at some point, you'll have to make the same changes to the patch for older series. When doing the rebase -i for the older series, replace the short sha of the change with the short sha of the new version of the change from the $SERIES-1.$X branch.