Ubuntu Wiki

   1 [19:01] <barry> hello everyone.  welcome to today's class on ubuntu distributed development (udd).  this is an hour-long class so hopefully we'll be able to get to everyone's questions.  i'll start here in just a minute.  don't forget to join #ubuntu-classroom-chat to ask your questions, prefaced with QUESTION:
   2 [19:03] <barry> okay, let's get started
   3 [19:03] <barry> first a little bit of background
   4 [19:03] <barry> here is the online documentation for udd. note however that it's slightly out of date at the moment.  i just merged a branch that updated a lot of information we're going to cover today, but it hasn't hit the web yet
   5 [19:04] <barry> http://developer.ubuntu.com/packaging/html/index.html
   6 [19:04] <barry> hopefully it will very soon now, but you can always grab the ubuntu packaging guide branch and build the html yourself.  ask a question in #u-c-c if you want details on that
   7 [19:05] <barry> i'm not really planning on covering packaging basics, hopefully you are all familiar with that.  i'm also not going to cover the traditional "apt-get source" way of doing packaging.  it's not strictly necessary that you know about that, but it might help for mapping the traditional way with the udd way.
   8 [19:05] <barry> if not, don't worry, you can pretty much do everything in udd these days
   9 [19:06] <barry> okay, so what is udd?
  10 [19:07] <barry> udd uses launchpad and the bazaar dvcs to do package development.  it has many advantages over the traditional apt-get source way (imho), and while there are still a few warts in the process, the tools are really fantastic these days thanks to the great bzr and lp teams
  11 [19:08] <barry> i'll also mention that most of what i'll discuss will be focused on using precise as the dev platform.  you can use oneiric, but you will want to get the latest bzr and bzr-builddeb packages from the bzr ppa if you do.  precise has some really great improvements that will make your life easier
  12 [19:08] <barry> so...
  13 [19:09] <barry> as you probably know, launchpad keeps bzr branches for all the packages in the ubuntu archive, and also all the packages in the debian archive.  we call these 'source branches'
  14 [19:10] <barry> lp runs a package importer process so whenever someone uploads a new version of a package, the importer will grab that and update the source branch for the package.  it works in 98% or so of cases.  the importer has some problems with a few packages, but at least now the tools will tell you if that's the case (and then you can drop down to the traditional way for those small number of problematic cases)
  15 [19:10] <barry> launchpad has source branches not just for the current in-development version of ubuntu and debian, but also for most older versions as well.  e.g. oneiric, lucid, squeeze, etc.
  16 [19:11] <barry> note that these source branches are distinct from any upstream branches lp may be tracking.  the source branches have the full unpacked source of the upstream as it exists in the ubuntu archive, along with the debian/ directory that is the packaging goodness.
  17 [19:12] <barry> the source branches also have the full history of the package, both in individual bzr revisions, and as bzr tags for easier access.
  18 [19:13] <barry> udd then is using these source branches to fix bugs, update to newer upstream versions, merge in new debian versions, build source packages for uploading, etc.  i.e. we now do everything with bazaar
  19 [19:13] <barry> any questions so far?
  20 [19:14] <barry> okay then
  21 [19:15] <barry> so let's say you want to fix a bug in tomboy using udd.  let's start by grabbing the source branch for tomboy in precise
  22 [19:16] <barry> i always find it useful to do my work in a bzr shared repo.  this amortizes the cost of doing the initial branching.  i personally like a shared repo per package
  23 [19:16] <barry> so, cd to someplace where you're going to do your work and then type `bzr init-repo tomboy`
  24 [19:16] <barry> then cd into the tomboy directory
  25 [19:17] <barry> bzr has a very nice shortcut for accessing the source branches for any package in the in-development version of ubuntu (i.e. precise).  just do this to grab the tomboy source branch:
  26 [19:17] <barry> bzr branch ubuntu:tomboy precise
  27 [19:18] <barry> you will end up with a `precise` directory containing the source branch
  28 [19:18] <barry> as the branching happens, notice two very useful things:
  29 [19:18] <barry> Most recent Ubuntu version: 1.8.0-1ubuntu1.2
  30 [19:18] <barry> Packaging branch status: CURRENT
  31 [19:18] <barry> bzr prints both those lines to your console
  32 [19:19] <barry> what that is telling you is that the most recent ubuntu version of tomboy (i.e. in precise) is 1.8.0-1ubuntu1.2 *and* that the packaging branch is up-to-date with the state of the archive
  33 [19:19] <barry> this is great because it means we can use udd to do our development
  34 [19:20] <barry> remember above i mentioned that the package importer sometimes has problems?  you would see a different message above if the tomboy branch was out of date
  35 [19:21] <ClassBot> jincreator asked: In common package, is maintainer the only one who can edit package?
  36 [19:22] <ClassBot> kanliot asked: i think i know what a bzr repo is, but what's a shared repo?
  37 [19:23] <barry> kanliot: a shared repo is just a parent directory that contains all the bazaar revision metadata.  it's nice to use because let's say you want to make a change to tomboy as described above.  when you do that in a shared repo, 99.9% of the revisions are going to be shared by your two branches.  all that shared metadata lives in the parent directory's .bzr directory.  so using a shared repo makes it super cheap to branch and merge
  38 [19:24] <barry> kanliot: hopefully that answers your question.  if not ask again, and i can dig up some bzr docos that explain it better
  39 [19:24] <barry> okay, depending on your bandwidth, you hopefully have the precise source branch for tomboy
  40 [19:25] <barry> i'll just mention a couple of things.  notice we used the ubuntu: prefix?  well, there's also a debianlp: prefix which can get the current debian version of the package (in wheezy, a.k.a. the in-development version)
  41 [19:25] <barry> if we wanted to get the lucid version of a package we would use
  42 [19:25] <barry> ubuntu:lucid/tomboy or abbreviated ubuntu:l/tomboy
  43 [19:26] <barry> abbreviations don't work for debian branches, but you can still do things like
  44 [19:26] <barry> debianlp:squeeze/tomboy to get the squeeze version of the package
  45 [19:27] <barry> okay, let's get back to our example
  46 [19:27] <barry> we've got the precise version of tomboy and now we want to create a branch to work on our fix.  let's say we're working on bug 12345.  we might do something like this:
  47 [19:27] <barry> bzr branch precise bug-12345
  48 [19:27] <barry> that should go really fast in a shared repo
  49 [19:28] <barry> go ahead and cd into bug-12345 and ls
  50 [19:28] <barry> you will see all the upstream source code, plus a debian directory
  51 [19:28] <barry> the debian dir contains all the packaging stuff of course
  52 [19:29] <barry> if we did nothing else, we could build a source package for local building and testing, by doing this:
  53 [19:29] <barry> bzr bd -S
  54 [19:29] <barry> this is roughly equivalent to debuild -S
  55 [19:29] <barry> we can also do normal package development stuff like
  56 [19:29] <barry> dch -i
  57 [19:29] <barry> and so on
  58 [19:29] <barry> a quick word about package systems
  59 [19:30] <barry> er, s/package/patch/
  60 [19:30] <barry> if you're an experienced packager you know that in general, we prefer to provide fixes to packages using a patch system, quilt being the most popular and recommended these days
  61 [19:31] <barry> the important thing to remember is that source branches already have all the patches applied
  62 [19:31] <barry> so once you get a source branch, you're ready to dig right in
  63 [19:31] <barry> it's equivalent to having already done `quilt push -a`
  64 [19:31] <barry> this generally makes things much nicer, but there are a few small gotchas we'll hopefully have time to get to
  65 [19:32] <barry> inside your bug-12345 branch, do this: quilt applied
  66 [19:32] <barry> that will show you that all the quilt patches in tomboy are already applied
  67 [19:32] <barry> let's say bug 12345 just described a typo in the debian/control file.  okay, this is easy to fix
  68 [19:32] <barry> you use your favorite editor to fix the typo
  69 [19:33] <barry> then you use dch -i to add an entry to debian/changelog
  70 [19:33] <barry> if you're happy with the change, you can then do:
  71 [19:33] <barry> bzr commit
  72 [19:34] <barry> what's nice about this is that your commit message will be taken from your changelog entry, so generally you can just use that directly
  73 [19:34] <barry> once you've committed your change, build your source package:
  74 [19:34] <barry> bzr bd -S
  75 [19:34] <barry> in the shared repo (i.e. the parent of bug-12345), you'll have the .dsc .orig.tar.gz and so on
  76 [19:34] <barry> you can test and upload these as normal
  77 [19:34] <barry> before you upload though, do one other thing:
  78 [19:34] <barry> bzr tag
  79 [19:35] <barry> (with no options).  this will add a tag matching the new changelog entry version number.  super easy!
  80 [19:35] <barry> you can then merge this back into the source branch by doing this:
  81 [19:35] <barry> cd ../precise
  82 [19:35] <barry> bzr merge ../bug-12345
  83 [19:35] <barry> bzr commit
  84 [19:36] <barry> bzr push :parent
  85 [19:36] <barry> that last bit pushes your updates back to launchpad (assuming you have permission to do so, i.e. you have upload rights on the package)
  86 [19:36] <barry> note that you'll still need to dput the package
  87 [19:36] <barry> what if you don't have upload rights?
  88 [19:37] <barry> in that case, you submit a merge proposal so that a package maintainer or sponsor can help you get your changes uploaded
  89 [19:37] <barry> in that case
  90 [19:37] <barry> stay in the bug-12345 directory.  after you've committed and tagged, do the following:
  91 [19:37] <barry> bzr push lp:~barry/ubuntu/precise/tomboy/bug-12345
  92 [19:37] <barry> substituting your own lp id for ~barry of course
  93 [19:38] <barry> the last path component is up to you.  i like naming it after the bug i'm fixing
  94 [19:38] <barry> once the branch is pushed, do the following:
  95 [19:38] <barry> bzr lp-propose
  96 [19:38] <barry> that will create a merge proposal with all the right details, and open your browser to that page
  97 [19:39] <barry> a sponsor can then review your changes, work with you to ensure they are correct, and help you get your changes uploaded
  98 [19:39] <barry> i'll pause a moment for questions
  99 [19:40] <barry> okay.  the next topic i'd like to cover is merging updates from debian or upstream using udd
 100 [19:41] <barry> let's say upstream tomboy releases a new version and you'd like to get that into ubuntu.  for whatever reason, you're going to do this without going through debian first
 101 [19:41] <ClassBot> fjrodriguez asked: What are upstream branches?
 102 [19:42] <barry> fjrodriguez: by "upstream branches" we can mean a couple of things.  because debian is an upstream of ubuntu, we could mean the version of the package in debian.  more typically though, we mean the code from the upstream project itself, although these will generally be tar.gz files or similar and not actual dvcs branches
 103 [19:42] <barry> (or non-d vcs branches)
 104 [19:43] <barry> i.e. when tomboy makes a new release, we're going to call the new  .tar.gz file the "upstream" version
 105 [19:43] <barry> does that make sense?
 106 [19:44] <barry> so let's say that tomboy releases version 2.0 and we want to put this in ubuntu (the following commands likely won't work since this is a hypothetical situation)
 107 [19:45] <barry> if the source branch, i.e. ubuntu:tomboy has a working debian/watch file, and we're lucky, we should just be able to do the following:
 108 [19:45] <barry> bzr merge-upstream
 109 [19:46] <barry> this command will grab the upstream tarball from the web, do a bunch of magic <wink>, and leave you with a branch of uncommitted changes that include an updated source code, and new debian/changelog entries for the new upstream version
 110 [19:46] <barry> everyone know what a debian/watch file is?
 111 [19:47] <barry> if the debian/watch file doesn't work, you can provide more options to bzr merge-upstream to help it along.  do `bzr merge-upstream --help` for details.  i'm going to skip ahead 'cause we have only 15 minutes left
 112 [19:47] <barry> kanliot: a debian/watch file contains a url pattern for how to find the upstream released tarball
 113 [19:48] <barry> the uscan command reads debian/watch and this can be used to find the latest released tarball (by checking version numbers).  `bzr merge-upstream` uses uscan under the hood to download that tarball
 114 [19:49] <barry> more common than getting a new upstream version into ubuntu is merging a new debian version of the package into ubuntu
 115 [19:49] <barry> jincreator: yes, debian/watch is packaging basics and not specifically udd related.
 116 [19:50] <ClassBot> There are 10 minutes remaining in the current session.
 117 [19:50] <barry> so let's say debian wheezy has tomboy 1.9 while ubuntu has tomboy 1.8.0.  we want to merge the debian version into the ubuntu version
 118 [19:50] <barry> in this case, let's assume that launchpad has the latest debian source branch
 119 [19:51] <barry> we can create a new local branch of ubuntu:tomboy, e.g.
 120 [19:51] <barry> bzr branch precise mergedeb
 121 [19:51] <barry> then cd in mergedeb and do this
 122 [19:51] <barry> bzr merge debianlp:tomboy
 123 [19:52] <ClassBot> kanliot asked: what are some specific scenarios where i would need bzr?  would it help if the debian source is downstream of an upstream source on sourceforge.net?
 124 [19:53] <barry> kanliot: one of the primary reasons for using bzr is if you want to work on packages for which you don't have upload rights.  by creating branches, pushing them to lp, and submitting merge proposals, you can gain street cred for your fixing, ubu devel, and packaging skills, and a sponsor can help you with the last little bit of getting your changes uploaded
 125 [19:53] <barry> does that make sense?
 126 [19:54] <barry> kanliot: lp is abbreviation for launchpad
 127 [19:54] <barry> it's the central hub for all ubuntu development
 128 [19:54] <barry> kanliot: correct.  bzr can be used in some cases for debian, but it's not as universal as it is for ubuntu
 129 [19:55] <ClassBot> There are 5 minutes remaining in the current session.
 130 [19:55] <barry> and debian has a different culture and workflow for package development, which i can't go into here ;)
 131 [19:55] <ClassBot> jincreator asked: Why Most recent Debian Version is "MISSING" when I merge from debian?
 132 [19:55] <barry> jincreator: that's an artifact of launchpad not knowing exactly what's in the debian archive.  i believe there is a bug open on this to try to improve that message
 133 [19:56] <barry> for now, you'll always get it when referring to debianlp: branches
 134 [19:57] <barry> well, i was hoping to get to using udd to work with quilt patches, because there's some really fantastic new stuff in precise about this, but there really isn't time.  i'll stick around #ubuntu-classroom-chat if you want to talk about that or you can always ping me on #ubuntu-devel
 135 [19:57] <barry> and the url for the packaging guide will have much better and simpler instructions on this soon
 136 [19:57] <ClassBot> benonsoftware asked: WHat CVS does Debian use Git, SVN, etc? (Sorry if I missed this part as I just came in :P)
 137 [19:58] <barry> benonsoftware: that's a little outside the scope of today's class, but different teams use different things.  e.g. the python packaging teams typically use svn
 138 [19:59] <barry> any other questions in our last 2 minutes?  my apologies for going so quickly and/or cutting things short due to time
 139 [19:59] <barry> let me add in closing that there are lots of places to learn about udd, ask questions, get help etc.
 140 [19:59] <barry> there is a udd mailing list: ubuntu-distributed-devel@lists.launchpad.net
 141 [20:00] <barry> i am always on #ubuntu-devel
 142 [20:00] <barry> #bzr can also be very helpful