Ubuntu Open Week - How to fix bugs in Ubuntu - James Westby and Daniel Holbach - Thu, Nov 5, 2009
(11:00:35 AM) dholbach: james_w: ready for part 2? (11:00:44 AM) james_w: yes (11:00:50 AM) dholbach: alrighty (11:00:55 AM) dholbach: do have a bug we could get started with? (11:01:41 AM) dholbach: just to answer a quick question first: (11:01:42 AM) dholbach: <jtniehof> QUESTION: (if you're not going to get to gpg later): is the Ubuntu WoT same as Debian? (i.e. do I need to get my keys signed by two different sets of people?) (11:01:50 AM) dholbach: jtniehof: it's very strongly encouraged (11:01:59 AM) dholbach: but as far as I know not enforced (11:02:25 AM) dholbach: also... if your gpg run is done now, please head to https://launchpad.net/people/+me/+editpgpkeys and add it to Launchpad (11:02:35 AM) dholbach: (that's not necessary for the example now, but very useful :-)) (11:03:04 AM) dholbach: james_w: you have a bug we can start with? (11:03:43 AM) james_w: want to start with https://bugs.launchpad.net/ubuntu/+source/xtux/+bug/454115 ? (11:04:10 AM) dholbach: sounds good :) (11:04:37 AM) dholbach: james_w: want to kick off the bug fixing frenzy? :) (11:05:30 AM) james_w: ok (11:05:46 AM) james_w: so, we first need to get the source package so that we can examine it and change it (11:06:08 AM) james_w: so, please move to a temporary directory and run: (11:06:12 AM) james_w: apt-get source xtux (11:06:20 AM) james_w: that will go off and download the source package for you (11:07:14 AM) james_w: you will see some messages about what it is doing (11:08:31 AM) james_w: but it basically leaves you with the unpacked source package in (11:08:56 AM) james_w: xtux-0.2.030306 (11:08:59 AM) james_w: so run: (11:09:02 AM) james_w: cd xtux-0.2.030306 (11:09:50 AM) james_w: and you can see a whole bunch of files that make up this package (11:10:06 AM) james_w: the packaging itself lives in the ./debian directory (11:10:22 AM) james_w: and there are a whole bunch more files there (11:10:33 AM) james_w: we won't look at what each of them do now (11:10:37 AM) james_w: just a couple (11:10:41 AM) james_w: so, debian/changelog (11:10:47 AM) james_w: this is the changelog of the packaging (11:11:19 AM) james_w: you can see changes that have been made (11:11:45 AM) james_w: debian/control is some information about the package (11:12:45 AM) james_w: it has things like the description, dependencies and the list (11:12:48 AM) james_w: and the like (11:13:15 AM) james_w: debian/copyright contains information on the license of the pakcage (11:13:44 AM) james_w: so, let's look at the bug (11:14:58 AM) james_w: so, the problem is: (11:15:00 AM) james_w: Removing xtux-client ... (11:15:00 AM) james_w: dpkg: error processing xtux-client (--remove): (11:15:00 AM) james_w: subprocess installed post-removal script returned error exit status 1 (11:15:08 AM) james_w: what does this mean? (11:16:45 AM) james_w: the "post-removal script" that it refers to is one that a set of scripts (11:17:04 AM) james_w: that a package can provide (11:17:40 AM) james_w: these scripts are run by dpkg as the package is installed or removed (11:17:59 AM) james_w: so, let's look at the one that is failing (11:17:59 AM) dholbach: a lot of stuff is done during that step (11:18:17 AM) dholbach: like updating the docs database, updating icon caches etc. (11:18:38 AM) dholbach: in most cases you don't need them, unless you want anything special to happen during the installation or removal (11:18:49 AM) dholbach: if you do want that to happen, you need to be extremely careful (11:18:52 AM) james_w: yes (11:19:02 AM) dholbach: everything else will botch the installation of your users (11:19:04 AM) dholbach: like this case (11:19:24 AM) dholbach: (all the regular steps are added automatically :)) (11:19:53 AM) james_w: open the file debian/xtux-client.postrm (11:20:25 AM) dholbach: the real beef of the file is: (11:20:27 AM) dholbach: rmdir /etc/ggz/clients /etc/ggz >/dev/null 2>&1 (11:20:29 AM) james_w: "postrm" means "post rm", or "post removal" (11:20:38 AM) james_w: so we know it is the script that we want (11:20:54 AM) dholbach: everything else is stuff you need in there to get the "regular actions" taking after the removal of packages (11:21:11 AM) dholbach: the call I mentioned above is problematic for a few reasons (11:21:32 AM) dholbach: 1) it suppresses any output which makes it harder for us to figure what the problem is ( >/dev/null 2>&1) (11:21:44 AM) dholbach: 2) it does not check if those directories are available (11:22:00 AM) dholbach: 3) rmdir will fail if those directories are not empty (11:22:07 AM) dholbach: james_w: anything I forgot? (11:22:57 AM) james_w: it should perhaps only does it's work on purge as well? (11:23:04 AM) dholbach: ah, good point (11:23:33 AM) dholbach: (the difference between apt-get remove <pkg> and apt-get remove --purge <pkg> (or dpkg -P)) (11:23:47 AM) dholbach: james_w: what do we do to fix it? (11:24:33 AM) james_w: so, to fix 1) (11:24:51 AM) james_w: we need to to change the redirection (11:24:59 AM) james_w: instead of rmdir /etc/ggz/clients /etc/ggz >/dev/null 2>&1 (11:25:09 AM) james_w: do rmdir /etc/ggz/clients /etc/ggz 2>&1 >/dev/null (11:25:23 AM) james_w: or just rmdir /etc/ggz/clients /etc/ggz >/dev/null (11:25:43 AM) james_w: to fix 2) (11:25:56 AM) james_w: we need to guard this by a check for the existence of the directory (11:26:08 AM) james_w: and 3) change the way we run rmdir (11:26:21 AM) james_w: so something like: (11:26:22 AM) james_w: for dir in /etc/ggz/clients /etc/ggz; do (11:26:31 AM) james_w: if [ -d $dir ]; then (11:26:39 AM) james_w: rmdir --ignore-fail-on-non-empty $dir; (11:26:47 AM) james_w: fi; (11:26:53 AM) james_w: done (11:27:29 AM) james_w: the "[ -d $dir]" checks that the directory exists before it is removed (11:27:52 AM) dholbach: (that's the test(1) command) (11:27:54 AM) james_w: the --ignore-fail-on-non-empty means that the command won't error if the directories are not empty (11:28:28 AM) james_w: so, please go ahead and change the line in the original file to those few lines (11:28:32 AM) dholbach: that looks good to me, so we just replace the original rmdir call and we're done? (11:29:17 AM) james_w: for that file I think? (11:29:27 AM) dholbach: yep, looks good (11:29:56 AM) dholbach: so apart from test building the package and more importantly: (11:29:58 AM) dholbach: _____ _____ ____ _____ ___ _ _ ____ _ (11:29:58 AM) dholbach: |_ _| ____/ ___|_ _|_ _| \ | |/ ___| | (11:29:58 AM) dholbach: | | | _| \___ \ | | | || \| | | _| | (11:29:58 AM) dholbach: | | | |___ ___) || | | || |\ | |_| |_| (11:29:58 AM) dholbach: |_| |_____|____/ |_| |___|_| \_|\____(_) (11:30:01 AM) dholbach: (11:30:08 AM) james_w: ok (11:30:11 AM) dholbach: we need to document our changes properly so people know what we did and why :) (11:30:16 AM) james_w: so, next step is to edit the changelog (11:30:17 AM) james_w: yes! (11:30:22 AM) james_w: want to take this bit? (11:30:34 AM) dholbach: sure (11:30:50 AM) dholbach: I use the dch -i command from the devscripts package (11:30:59 AM) dholbach: this makes use of the DEBEMAIL variables we set before (11:31:12 AM) dholbach: and will add a template changelog entry for us and increment the version number (11:31:38 AM) dholbach: after some editing the first line for me now says (11:31:39 AM) dholbach: xtux (0.2.030306-11ubuntu1) lucid; urgency=low (11:31:44 AM) dholbach: let's go through it one by one (11:31:53 AM) dholbach: first up is the name of the source package (11:32:10 AM) dholbach: xtux-client (the binary ".deb" package) belongs to the xtux source package (11:32:21 AM) dholbach: one source package can build multiple binary packages (11:32:30 AM) dholbach: this is used to split things up for various reasons (11:32:52 AM) dholbach: this example will show you a package that makes use of the idea extensively: apt-cache showsrc mono (11:33:08 AM) dholbach: next up is the version number, which in our case now is: 0.2.030306-11ubuntu1 (11:33:44 AM) dholbach: it means that the software authors released 0.2.030306 at some stage which has gone through 11 revisions in Debian (11:33:51 AM) dholbach: and we're just about to add the first change to it in Ubuntu (11:34:10 AM) dholbach: next comes the distro release we want to upload it to later on (11:34:24 AM) dholbach: this should always default to the current development release which since some days is lucid (11:34:42 AM) dholbach: if you want to get something included in an older release, you might want to have a look at https://wiki.ubuntu.com/StableReleaseUpdates (11:35:28 AM) dholbach: we can ignore "urgency=low" - that's a Debian'ism that Launchpad does not make use of as far as I know (11:35:46 AM) dholbach: -- Daniel Holbach <firstname.lastname@example.org> Thu, 05 Nov 2009 17:30:38 +0100 (11:35:53 AM) dholbach: is the last line which isn't particularly interesting (11:35:54 AM) dholbach: :-) (11:36:01 AM) dholbach: so, let's document what we did (11:37:03 AM) dholbach: james_w might have a better wording than I do, but I'd put something like this in there (11:37:05 AM) dholbach: * xtux-client.postrm: don't fail to remove directories which don't exist (11:37:05 AM) dholbach: or are not empty (LP: #454115) (11:37:18 AM) dholbach: this does 3 things: (11:37:26 AM) dholbach: it mentions which files we changed (11:37:33 AM) dholbach: it mentions how we changed them (11:37:48 AM) dholbach: and it references the bug report in Launchpad we attempt to fix (11:38:01 AM) dholbach: (and it also closes the bug automatically when the package is uploaded) (11:38:06 AM) dholbach: this is important for a few reasons (11:38:19 AM) dholbach: we all work on Ubuntu packages togethere (11:38:29 AM) dholbach: there is no strict "owner" of a package (11:38:39 AM) dholbach: so you don't want others to have to second-guess what you actually changed (11:38:41 AM) dholbach: AND (11:38:50 AM) dholbach: YOU don't want to second-guess half a year later :-) (11:39:06 AM) dholbach: once you've done that, please save the file (11:39:23 AM) dholbach: james_w: anything I forgot before we crack on? (11:39:40 AM) james_w: don't think so (11:40:07 AM) dholbach: I stand correct: maco points out that urgency DOES make a difference in Ubuntu too: https://help.launchpad.net/Packaging/BuildScores (11:40:14 AM) dholbach: please use it wisely :) (11:40:23 AM) dholbach: alright, we should be done now (11:40:31 AM) dholbach: please run (11:40:34 AM) dholbach: debuild -S (11:40:45 AM) dholbach: this will rebuild the source package for us (11:41:20 AM) dholbach: (if you ran into an error saying something about Ubuntu maintainer, please run update-maintainer from ubuntu-dev-tools and debuild -S again) (11:41:47 AM) dholbach: (it also uses quilt, so if you get an error about that, please install quilt too) (11:42:45 AM) dholbach: once it completed I get the following files (below the xtux-0.2.0.030306 directory): (11:42:46 AM) dholbach: xtux_0.2.030306-11.diff.gz xtux_0.2.030306-11ubuntu1_source.build (11:42:46 AM) dholbach: xtux_0.2.030306-11.dsc xtux_0.2.030306-11ubuntu1_source.changes (11:42:46 AM) dholbach: xtux_0.2.030306-11ubuntu1.diff.gz xtux_0.2.030306.orig.tar.gz (11:42:46 AM) dholbach: xtux_0.2.030306-11ubuntu1.dsc (11:43:13 AM) dholbach: xtux_0.2.030306.orig.tar.gz contains the code that was actually released by the xtux authors (11:43:40 AM) dholbach: xtux_0.2.030306-11.dsc and xtux_0.2.030306-11.diff.gz basically make up the packaging by the Debian maintainer (of the -11 version) (11:43:54 AM) dholbach: xtux_0.2.030306-11ubuntu1.diff.gz and xtux_0.2.030306-11ubuntu1.dsc is -11 plus our good work (11:44:14 AM) dholbach: the .diff.gz files contain the compressed set of changes we need to apply to make the package build the "Debian/Ubuntu way" (11:44:20 AM) dholbach: the .dsc is metadata like md5sums (11:44:49 AM) dholbach: there's a nifty tool called debdiff, which we'll use here to get us the patch for the changes we just did (11:45:01 AM) dholbach: please run: (11:45:03 AM) dholbach: debdiff xtux_0.2.030306-11.dsc xtux_0.2.030306-11ubuntu1.dsc (11:46:19 AM) dholbach: the output should be a bit like this one: (11:46:20 AM) dholbach: http://paste.ubuntu.com/310719/ (11:46:43 AM) dholbach: if you got this far: you just created your first patch (11:46:47 AM) dholbach: CONGRATULATIONS! :)))) (11:46:55 AM) dholbach: james_w: want to take us from here? (11:48:04 AM) james_w: what's next? (11:48:13 AM) dholbach: we could test-build it (11:48:19 AM) dholbach: or try to squeeze in another bug :) (11:48:21 AM) james_w: of course! (11:48:33 AM) james_w: pbuilder build xtux_0.2.030306-11ubuntu1.dsc (11:48:42 AM) james_w: that will build your new package inside pbuilder (11:48:57 AM) dholbach: you might want to add a sudo in front :) (11:49:29 AM) dholbach: ok, let's hear some more questions (11:49:44 AM) dholbach: for those of you who want more bugs and more bug fixing action, you could try having a look at these bugs: (11:49:45 AM) dholbach: http://bugs.launchpad.net/ubuntu/+source/fet/+bug/368017 (11:49:45 AM) dholbach: https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/429986 (11:49:45 AM) dholbach: https://bugs.launchpad.net/ubuntu/+source/ibus/+bug/429988 (11:49:45 AM) dholbach: https://bugs.launchpad.net/ubuntu/+source/usermode/+bug/461365 (11:50:13 AM) dholbach: and check out https://wiki.ubuntu.com/MOTU/TODO which lists a lot of good stuff to get started with (11:50:35 AM) dholbach: do we have more questions? (11:51:16 AM) dholbach: did pbuilder succeed for anybody already? :) (11:51:43 AM) dholbach: <jsgruber> QUESTION: Is it reasonable to use your ppa rather than using pbuilder to test build a package? (11:51:48 AM) dholbach: jsgruber: good one (11:52:26 AM) dholbach: so jsgruber is referring to PPAs (personal package archives), a service by Launchpad to build your package for you: https://help.launchpad.net/PPA has more details about that (11:52:45 AM) dholbach: I usually prefer to build them locally - it's mostly quicker because you don't have a queue :) (11:53:05 AM) dholbach: I just use PPAs to distribute packages for testing (11:53:13 AM) dholbach: <openweek7> QUESTION: What if the bug fix you create fixes the bug you intended to fix, but creates other bugs in the source? How would you test for this? (11:53:37 AM) dholbach: openweek7: in our case, I'd test-install the package and see if I can remove it safely again (11:53:59 AM) dholbach: it's really important that you install the packages you just built and play around with them to make sure the fix indeed does what you promised (11:54:07 AM) dholbach: <Jesi-Idle> Question: what are the differences from building the "Debian/Ubuntu way" or the "Fedora" or "Suse" way since you mentioned it, just little things or is there a significant difference? (11:54:31 AM) dholbach: I won't go into much speculation about how it works for Fedora or SuSE because I don't really know (11:54:48 AM) dholbach: I know that in the .rpm world a .spec file is involved which is a condensed ./debian directory (11:55:05 AM) mode (+v mneptok ) by jcastro (11:55:34 AM) dholbach: what I can say is that the debian/ubuntu way of building software basically means that you wrap a well-defined build system around that that comes from the upstream software (11:55:40 AM) dholbach: I know that's a bit "meta" :) (11:56:03 AM) mode (+v jussi01 ) by jcastro (11:56:06 AM) dholbach: a lot of C programs usually take you a ./configure && make && sudo make install route (11:56:20 AM) dholbach: python projects might require a python ./setup.py build <something> (11:56:31 AM) dholbach: and there's lots of other software that does it in different ways (11:56:46 AM) dholbach: we have clever tools for all the "regular" cases (11:56:59 AM) dholbach: so you merely run fakeroot debian/rules clean (11:57:05 AM) dholbach: or fakeroot debian/rules binary (11:57:08 AM) dholbach: or whatever (11:57:32 AM) dholbach: basically a build system with build targets around the one you're required to follow by that software (11:57:42 AM) dholbach: <ia> QUESTION: is it reasonable updating system (using for testing devel releases and ubuntu devel purposes) to lucid (before alpha1 release) today? or should better to wait for alpha1? (11:57:56 AM) dholbach: I'd encourage you to read https://wiki.ubuntu.com/UbuntuDevelopment/UsingDevelopmentReleases (11:58:13 AM) dholbach: because it explains how to set up lucid in a chroot or a vm or a separate partition or something (11:58:22 AM) dholbach: and you don't have to deal with the daily pain :-) (11:58:37 AM) dholbach: alrightie (11:58:46 AM) dholbach: please bookmark: (11:58:46 AM) dholbach: ¡¡¡ https://wiki.ubuntu.com/MOTU/GettingStarted !!! (11:58:51 AM) dholbach: see you in #ubuntu-motu (11:58:57 AM) dholbach: and make james_w and me proud! (11:59:00 AM) dholbach: thanks a lot james_w (11:59:06 AM) dholbach: and thanks a lot everybody for attending the session!