Ubuntu Wiki

   1 [17:00] <tumbleweed> Hello everyone
   2 [17:00] <tumbleweed> thanks coolbhavi, hope you all enjoyed is session
   3 === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Developer Week - Current Session: Fixing small bugs and forwarding the patches upstream - Instructors: tumbleweed
   4 [17:00] <tumbleweed> *his
   5 [17:00] <ClassBot> Logs for this session will be available at http://irclogs.ubuntu.com/2012/08/29/%23ubuntu-classroom.html following the conclusion of the session.
   6 [17:01] <tumbleweed> everyone having a good Ubuntu Developer Week?
   7 [17:01] <tumbleweed> Let's get an idea of how many people are here: please say hi in the chat channel
   8 [17:01] <tumbleweed> that's #ubuntu-classroom-chat, which is also the place to ask questions. Please start them with QUESTION: so the bot can pick them up
   9 [17:01] <tumbleweed> If you have any questions with any of this, when you go home and do this, afterwards
  10 [17:01] <tumbleweed> I suggest sticking your nose into #ubuntu-motu and asking there
  11 [17:01] <tumbleweed> it's full of friendly ubuntu developers
  12 [17:01] <tumbleweed> \o/ Glad to see such good attendance at these talks
  13 [17:02] <tumbleweed> so, I'm Stefano Rivera, an Ubuntu Developer and a Debian Developer
  14 [17:02] <tumbleweed> I live in (the not quite so sunny at the moment) South Africa
  15 [17:03] <tumbleweed> (and right now, I'm in a friends flat, with a laptop, in the corner, on a 3G connection. So hopefully I won't unexpectedly vanish)
  16 [17:03] <tumbleweed> I'm here to talk about dealing with upstreams
  17 [17:03]  * tumbleweed sees a few familiar faces here :P
  18 [17:03] <tumbleweed> What do I mean by an upstream?
  19 [17:03] <tumbleweed> well, Ubuntu is made up of ~20 000 source packages, (which build ~40 000 binary packages)
  20 [17:03] <tumbleweed> us Ubuntu Developers don't write all of those ourselves. We only authored a tiny handful of them
  21 [17:04] <tumbleweed> everything else came from other projects
  22 [17:04] <tumbleweed> some of these are big projects like GNOME and KDE, which produce hundreds of packages
  23 [17:04] <tumbleweed> most are tiny little projects with one, or maybe half a dozen active contributors
  24 [17:04] <tumbleweed> these projects are our upstreams
  25 [17:04] <tumbleweed> they release new versions of their software, and we package it, make a few tweaks to get it to work nicely with the rest of Ubuntu, and ship it to users
  26 [17:04] <tumbleweed> then we are on the front-line of incoming bug reports, and have to forward the relevant ones back to the upstreams
  27 [17:04] <tumbleweed> sounds like fun, right :)
  28 [17:05] <tumbleweed> OK, I lied a bit there
  29 [17:05] <tumbleweed> I assume you all know that Ubuntu is derived from Debian
  30 [17:05] <tumbleweed> in fact, about 75% of packages in Universe are entirely unmodified from Debian
  31 [17:05] <tumbleweed> that's a very good thing
  32 [17:05] <tumbleweed> there's no need for those packages to be modified in Ubuntu
  33 [17:05] <tumbleweed> the vast majority of bugs that we'd encounter in them are relevant to Debian, Ubuntu, and (if except for packaging bugs, the original upstream)
  34 [17:05] <tumbleweed> So Debian is also an upstream of Ubuntu. A very special upstream which we care a lot about
  35 [17:06] <tumbleweed> We contribute a lot back to Debian (as much as possible)
  36 [17:06] <tumbleweed> https://wiki.ubuntu.com/Debian/ForUbuntuDevelopers
  37 [17:06] <tumbleweed> I can go into some more specifics of dealing with Debian, later
  38 [17:06] <tumbleweed> but, now, let's talk about upstreams in general
  39 [17:06] <tumbleweed> it makes sense to fix bugs them as close to the source as possible
  40 [17:07] <tumbleweed> that means the least duplication of effort, and shares the benefit as widely as possible
  41 [17:07] <ClassBot> marcosb asked: please tell us about packages nameclature, like hello_1.1-1ubuntu1.deb hello_1.1-1upstream1.deb etc.
  42 [17:07] <tumbleweed> sure
  43 [17:08] <tumbleweed> all .deb files are named package_version_arch.deb
  44 [17:08] <tumbleweed> the version can give some idea of the package's ancestory
  45 [17:09] <tumbleweed> packages that were written in Debian, only for Debian (and derivatives) have a version that looks like 1.0.0
  46 [17:09] <tumbleweed> there's no '-1' at the end
  47 [17:09] <tumbleweed> if a debian developer packages an usptream version 1.0.0, her package will be versioned 1.0.0-1
  48 [17:09] <tumbleweed> -1 meaning first upload to Debian
  49 [17:09] <tumbleweed> if we modify that package in Ubuntu, the version would be 1.0.0-1ubuntu1
  50 [17:10] <tumbleweed> if we jumped to 1.0.1 in Ubuntu, before Debian got there, we'd call it 1.0.1-0ubuntu1
  51 [17:10] <tumbleweed> etc.
  52 [17:11] <tumbleweed> of course, not everyone follows these rules, but almost all the time, they are relevant
  53 [17:12] <tumbleweed> so, I was talking about why we should work with upstreams:
  54 [17:12] <tumbleweed> if we submit a patch to the upstream developers, and they release a new release containing our bug fix, it benefits everyone:
  55 [17:12] <tumbleweed> users of other linux distributions, cygwin and macports users, people who installed this package themselves from scratch, etc.
  56 [17:12] <tumbleweed> and the upstream can probably give you useful review of your bug fix
  57 [17:12] <tumbleweed> nobody knows the code like they do
  58 [17:13] <tumbleweed> I hope at this point, you are aware of some the advantages of working well with our upstreams
  59 [17:13] <tumbleweed> (and that I might have written up some of this beforehand, for pasting) :P
  60 [17:14] <tumbleweed> How can you find a bug to work on?
  61 [17:14] <tumbleweed> We've tried to put a good collection of things for new developers to work on here:
  62 [17:14] <tumbleweed> https://wiki.ubuntu.com/UbuntuDevelopment/BugFixingInitiative
  63 [17:14] <tumbleweed> The first batches there are Ubuntu-only issues, which won't require any forwarding to anyone
  64 [17:14] <tumbleweed> So they're a good place to start if you Ubuntu Development is something you want to get into
  65 [17:14] <tumbleweed> The later batches affect more than just Ubuntu, and the patches for them will need to be sent upstream
  66 [17:14] <tumbleweed> (which is what we are really here to talk about today)
  67 [17:15] <tumbleweed> Another good way to get into Ubuntu Development is to find a package that you care about, that needs work
  68 [17:15] <tumbleweed> Something with a few (or even lots of) open bugs that really just needs someone to go through them
  69 [17:15] <tumbleweed> maybe we've even fixed a few bugs in it, and not sent the patches upstream
  70 [17:16] <tumbleweed> Ubuntu developers can be naughty and lazy :)
  71 [17:16] <tumbleweed> There are *lots* of packages like that in the archive
  72 [17:16] <tumbleweed> In Debian, every package has explicit Maintainers who are looking after them
  73 [17:16] <tumbleweed> But in Ubuntu, most packages don't have anyone paying any attention to them
  74 [17:16] <tumbleweed> I've tried to collect a list of especially neglected packages that aren't in Debian, here:
  75 [17:16] <tumbleweed> http://qa.ubuntuwire.org/neglected/
  76 [17:16] <ClassBot> AmberJ_ asked: Quoting you from #ubuntu-classroom "if we modify that package in Ubuntu, the version would be 1.0.0-1ubuntu1" ... what if we don't modify Debian package in Ubuntu?
  77 [17:16] <tumbleweed> then it keeps the same version as Debian
  78 [17:17] <tumbleweed> as I said much earlier, most of the archive is entirely unmodified debian
  79 [17:17] <tumbleweed> we have a slightly different toolchain that'll build things a little differently
  80 [17:17] <tumbleweed> but most packages don't have modified source
  81 [17:18] <tumbleweed> How do you fix the bug?
  82 [17:18] <tumbleweed> That's a hard question to answer. It depends entirely on the bug
  83 [17:18] <tumbleweed> All I can really tell you about is how to get your hands on the source code and what to do once you've fixed it
  84 [17:18] <tumbleweed> dholbach will have covered the first part o fthat, yesterday
  85 [17:18] <tumbleweed> but let me remind you
  86 [17:19] <tumbleweed> install the packaging-dev package, it'll give you lots of useful tools
  87 [17:19] <tumbleweed> you can grab the source for an Ubuntu package by running: pull-lp-source PACKAGENAME
  88 [17:20] <ClassBot> marcosb asked: I learn how create patches in this session?
  89 [17:20] <tumbleweed> sure, that sounds like a useful thing to talk about
  90 [17:20] <tumbleweed> there are two common ways of dealing with source packages in Ubuntu. The traditional debian tools, and bzr
  91 [17:21] <tumbleweed> I'm going to mostly talk about the traditional tools because they are more widely applicable, and I find them faster
  92 [17:22] <tumbleweed> so, you've downloaded and extracted a source package with pull-lp-source
  93 [17:22] <tumbleweed> now, you see something obvious to fix
  94 [17:23] <tumbleweed> at this point, there could be several different things you need to do, thanks to a few different ways of packaging things
  95 [17:23] <tumbleweed> you can run the "what-patch" command to see how the package represents patches
  96 [17:24] <tumbleweed> let's assume that the package uses quilt patches, they are the most common these days. Hopefully the other systems will fade away
  97 [17:24] <tumbleweed> you'd run: quilt new MYPATCHNAME
  98 [17:25] <tumbleweed> then quilt edit FILENAME
  99 [17:25] <tumbleweed> then quilt refresh, which will build the patch
 100 [17:25] <tumbleweed> and you can find it in debian/patches
 101 [17:26] <tumbleweed> most of the time, when working on debian packages, that's how you'll be generating the patches
 102 [17:27] <tumbleweed> that patch you generated, you can now send to the upstream project
 103 [17:27] <tumbleweed> how do you find where they are?
 104 [17:27] <tumbleweed> well, let's start with Debian
 105 [17:28] <tumbleweed> every You can see more information about the package at http://packages.qa.debian.org/$PACKAGENAME
 106 [17:28] <tumbleweed> err ignore the every
 107 [17:28] <tumbleweed> so, for the hello package, go to http://packages.qa.debian.org/hello
 108 [17:28] <tumbleweed> at this point, I'm talking about source packages, not binary packages
 109 [17:28] <tumbleweed> as developers, we're almost entirely working with source pcakages
 110 [17:29] <tumbleweed> the binary packages are just smoething that'll be built when we are done
 111 [17:29] <tumbleweed> on that packages.qa.debian.org page (called the PTS, package tracking system)
 112 [17:30] <tumbleweed> you'll find a list of the debian bugs, in the top rigth corner
 113 [17:30] <tumbleweed> you can see who the debian maintainer is in the top left
 114 [17:30] <tumbleweed> and see the upload history in the middle
 115 [17:31] <tumbleweed> so, now we'd look through the debian bugs
 116 [17:31] <tumbleweed> there's currently only one
 117 [17:31] <tumbleweed> but if we saw the bug we were working on, there, we'd go and have a look at it, and see what progress there has been
 118 [17:32] <tumbleweed> maybe there's a patch on it? maybe it's been forwarded to the upstream
 119 [17:32] <tumbleweed> we can see that the bug there (bugs.debian.org/621716) has been forwarded upstream
 120 [17:32] <tumbleweed> we don't have a link to the upstream bug, though, it was just forwarded to an e-mail address
 121 [17:33] <tumbleweed> some usptream projects don't have formal bug trackers, and the best we can do is say "we mailed the author"
 122 [17:33] <ClassBot> marcosb asked: there's another patch mechanisms or just quilt?
 123 [17:34] <tumbleweed> yeah, I'm afraid so. there are two other common patch systems: dpatch and cdbs-simple-patch (but both are declining in popularity)
 124 [17:35] <tumbleweed> and some packages have no patch management system at all
 125 [17:35] <tumbleweed> that means that all the patches that debian & ubuntu have applied to the package are mixed together in one big diff
 126 [17:35] <tumbleweed> that's really easy to work with, but hard to understand what's going on in the patch
 127 [17:37] <ClassBot> calmi asked: is there a system that helps to avoid duplicates?
 128 [17:37] <tumbleweed> yes, look before you file new bugs
 129 [17:37] <tumbleweed> launchpad's bug filing interface is actually really good at this
 130 [17:37] <tumbleweed> after you've entered a title, it finds you similar bugs
 131 [17:38] <tumbleweed> the standard bug filing tool for Debian, reportbug, also presents a list of all open bugs for you to review, before filing yoru bug
 132 [17:38] <ClassBot> marcosb asked: Can you tell how generate packages with dpatch and dbs-simple-patch?
 133 [17:38] <tumbleweed> I won't cover those now, they are rare enough that we don't need to worry about them (I think)
 134 [17:38] <tumbleweed> let's rather just talk about packages that don't have any patch system at all
 135 [17:39] <tumbleweed> the easiest way to get a patch out of them
 136 [17:39] <tumbleweed> is to make your change, add a new changelog entry with dch (I assume dholbach covered that earlier)
 137 [17:39] <tumbleweed> and build a source package
 138 [17:39] <tumbleweed> then you can diff the old package to the new one with the "debdiff" tool
 139 [17:40] <tumbleweed> so, debdiff foobar_0.1-1.dsc foobar_0.1-1ubuntu1.dsc
 140 [17:41] <tumbleweed> and you'll get a patch that shows the difference between 0.1-1 and 0.1-1ubuntu1
 141 [17:42] <tumbleweed> (sorry, lost 3g signal for a minute, there)
 142 [17:42] <tumbleweed> back to what I was saying before:
 143 [17:43] <tumbleweed> so, we've been looking at the debian bug tracker, and we don't see a bug that describes the issue we ware working on
 144 [17:43] <tumbleweed> now, we make a decsion: is the bug I'm working on a debian packaging bug?
 145 [17:43] <tumbleweed> if so, we'd better forward it to debian
 146 [17:44] <tumbleweed> is it an upstream bug, but something that's important enough that we should fix it in Debian while we wait for the upstream to fix it
 147 [17:44] <tumbleweed> or it affects a debian stable release, and is serious enough that they'd want to cherry-pcik the fix for the stable release
 148 [17:44] <tumbleweed> if so, we'd also forward it to debian
 149 [17:45] <tumbleweed> if we don't have a reason to forward it debian, we are just looking for the upstream authors
 150 [17:45] <tumbleweed> we can find them in a few ways
 151 [17:45] <tumbleweed> most packages helpfully have a Homepage field in the control file, tha ttells us where the upstream project is
 152 [17:45] <tumbleweed> if there is such a tag, you'll see a link to the homepage on the right side of the PTS
 153 [17:46] <tumbleweed> e.g the hello package has such a link: http://packages.qa.debian.org/hello
 154 [17:46] <tumbleweed> taking us to http://www.gnu.org/software/hello/
 155 [17:46] <tumbleweed> and that page tells us that the hello bugtracker can be found on savannah http://savannah.gnu.org/projects/hello/
 156 [17:46] <tumbleweed> eventually, we got there.
 157 [17:47] <tumbleweed> now, we can file a bug with them (after seeing if it's already there, or been fixed already) and giv them the patch we created
 158 [17:48] <tumbleweed> \o/ success
 159 [17:48] <ClassBot> jsjgruber-l85-p asked: Might not the upstream ask us things we don't know about how the original bug reporter found the bug, etc.?
 160 [17:48] <tumbleweed> that's very likely
 161 [17:48] <tumbleweed> many upstreams aren't particularly interested in linux distributions that ship "ancient" versions of their packages to users
 162 [17:49] <tumbleweed> so, the most useful thing you can do is see if the the current development version of their package still has the issue
 163 [17:49] <tumbleweed> how you do that will depend entirely on the project
 164 [17:49] <tumbleweed> I generally find it useful to reproduce the bug myself
 165 [17:49] <tumbleweed> then I know if I've fixed it or not, for a start
 166 [17:50] <tumbleweed> and I can place myself in the same position as the original reporter
 167 [17:50] <tumbleweed> and write a good bug report for the upstream
 168 [17:50] <tumbleweed> if only every bug was easily reproduced, though :P
 169 [17:50] <tumbleweed> any more questions? we have 10 mins left
 170 [17:50] <ClassBot> There are 10 minutes remaining in the current session.
 171 [17:51] <tumbleweed> oh, snap, ClassBot
 172 [17:51]  * tumbleweed gives ClassBot a cookie
 173 [17:52] <tumbleweed> ok, well in that case, let me spend a few minutes talking about the debian bugtracking system
 174 [17:52] <tumbleweed> I'll happily be interrupted by questions, though
 175 [17:52] <ClassBot> jsjgruber-l85-p asked: What can you do if the Debian/upstream developer doesn't get back to you?
 176 [17:53] <tumbleweed> we can fix the issue in Ubuntu, and get on with lifet
 177 [17:53] <tumbleweed> *life
 178 [17:53] <tumbleweed> if the upstreams aren't responding, to anything, though, that may be a sign that they are no longer usefully maintaining their package
 179 [17:53] <tumbleweed> if they stop responding at all, and stop producing new releases, after a while, we'd remove the package from debian/ubuntu
 180 [17:54] <tumbleweed> (unless someone else takes over the upstream development)
 181 [17:54] <tumbleweed> so, I still have a couple of minutes, back to the BTS
 182 [17:54] <tumbleweed> debian's bug tracker (the BTS) lives at bugs.debian.org
 183 [17:55] <tumbleweed> the easy links for you are: bugs.debian.org/PACKAGE, and bugs.debian.org/BUG
 184 [17:55] <ClassBot> There are 5 minutes remaining in the current session.
 185 [17:55] <tumbleweed> you'll notice that there aren't any forms for adding comments or filing bugs
 186 [17:55] <tumbleweed> that's because the whole thing is driven by email
 187 [17:56] <tumbleweed> you may find that a little archaic, but it works fairly well
 188 [17:56] <ClassBot> ben72 asked: When the patch is very small, does it make sense to just post it as a comment in the bugtracker?
 189 [17:57] <tumbleweed> well, yes
 190 [17:57] <tumbleweed> that's pretty much how we forward patches
 191 [17:57] <tumbleweed> hopefully the bug trackre has some way of attaching a patch to a comment
 192 [17:57] <tumbleweed> but sometimes you just have to include the patch inline in the comment
 193 [17:57] <tumbleweed> and sometimes the patch is so trivial, that you just have to describe it and don't need a patch
 194 [17:58] <tumbleweed> if your patch is too big: you are probably better of breaking it up into multiple patches
 195 [17:58] <tumbleweed> split into logical ideas
 196 [17:58] <ClassBot> marcosb asked: after a add changelog entry and patch with quilt, i generate new source (debuild -us -uc) and thendebdiff?
 197 [17:58] <tumbleweed> debuild -uc -us -S
 198 [17:58] <tumbleweed> then debdiff
 199 [17:59] <tumbleweed> note the -S which builds a source package only, no binary packages
 200 [17:59] <tumbleweed> (it's *much* quicker to not build binaries)
 201 [17:59] <tumbleweed> ok, I guess I'm out of time
 202 [17:59] <tumbleweed> thanks everyone
 203 [17:59] <tumbleweed> hope to see you all getting involved, soon