QA

Dev Week --Me, myself & QA -- warp10, gaspa, BlackZ -- Fri, Jul 16th, 2010

(04:00:31 PM) warp10: Thank you highvoltage, and hi everybody, guys! Who is here to have great moments learning about QA stuff, raise your hand (in #ubuntu-classroom-chat, please)!
(04:01:58 PM) warp10: OK, great. I'm Andrea Colangelo, an Ubuntu Developer. BlackZ, gaspa and me from the Ubuntu Italian Mafia Famiglia [Copyright (C) Daniel Holbach] will try to show you that QA is not that boring at all, and rather it is a very appreciated activity in Ubuntu development.
(04:02:41 PM) highvoltage left the room (quit: Quit: bbl).
(04:02:53 PM) warp10: In case you don't know already, QA means Quality Assurance, and it involves a number of procedures and tools aiming to keep Ubuntu packages and the whole archive in good shape.
(04:03:13 PM) warp10: Wrong or circular dependencies, packages that Fails To Build From Source (FTBFS) and that Not Build from Source (NBS) anymore, install and upgrade issues, fixing lintian errors: all of them are problems that QA activites are intended to fix.
(04:03:38 PM) warp10: QA is not focused on a particular set of packages, rather the whole archive is the domain of our activities. Therefore, a good committement to QA also gives you the possibility to increase your packaging knowledge and skills, and will allow you to reach the deeper and more obscure corners of the archive.
(04:04:08 PM) warp10: Further, a good QA also means working side-by-side with your fellow developers who packaged and/or cared of the package you are working on, and will give you that extraordinary feeling of "standing on the shoulders of giants" that new contributors well know.
(04:04:41 PM) warp10: And please, don't think QA involves Ubuntu only. Very often your fixes for Ubuntu issues apply to Debian too, and a good Ubuntu Developer is always ready to open reportbug and send patches upstream to Debian.
(04:05:01 PM) warp10: Therefore, QA and community are thigtly related, and a good QA will make all Ubuntu users happier. This is one among the many reasons why we like QA so much! :)
(04:05:34 PM) warp10: Since QA is such a wide and extensive field of activities, you will find *lot* of things to do, and *lot* of issues to fix anytime. But don't be scared: lots of tools have been deployed to help your efforts, tools that gaspa, BlackZ and me will try to show you in these remaining 55 minutes.
(04:06:06 PM) warp10: If you are really interested in QA, the website you will definitely want to add to your favourites is http://qa.ubuntu.com/. There you will find (almost) everything that is QA-related.
(04:06:34 PM) warp10: Lots of (hopefully) good words so far, but we still didn't saw anything. So, let's dirt our hands with something more pragmatic. But first of all: any question so far? (In -chat, please)
(04:07:20 PM) warp10: Ok, good. The first argument we will introduce is NBS. Who of you already knows what a NBS is? Raise your hands again (in -chat, please)!
(04:08:19 PM) warp10: Ok, great. NBS are one of the most appreciated QA activities, and the one I like most, personally. :)
(04:08:47 PM) warp10: NBS stands for "Not Built From Source". As you can easily understand, it refers to packages that, altough still in the archive, are no longer built from source.
(04:09:11 PM) warp10: You might wonder why such a situation could happen. Well, there is a number of different reasons. The most common one is that the package has been renamed.
(04:09:46 PM) warp10: For examble, upstream released a brand new release and renamed his software, so we may want to rename the package as well. Another possibility is that a library gets a new SONAME and this has to be reflected in the binary name.
(04:10:12 PM) warp10: (For people who don't know already, a SONAME is a sort of "name" describing what functionalities a certain library or piece of software exposes. See also: http://en.wikipedia.org/wiki/Soname.)
(04:10:44 PM) warp10: If there are packages that still depend on the NBS package, the NBS can't be removed from the archive, or it would cause an unmetdep (i.e.: a package is not installable because one of its dependency is missing from archives).
(04:11:10 PM) warp10: Our work as QA guys is to carefully check the situation and rebuild the packages depending on our NBS, so that their dependencies list is updated with the new package, and then we can drop the useless NBS from archives.
(04:11:40 PM) warp10: Feeling a little confused? Yeah, comprehensible. Don't worry: we will see some examples in a few seconds. Also, please refer to https://wiki.ubuntu.com/UbuntuDevelopment/NBS for a broader overview and more examples. And ask your questions anytime in -chat if needed, ok?
(04:12:07 PM) warp10: The most important thing to understand right now is that when you tackle a certain NBS you won't work on the NBS package itself (it's a zombie! It is going to be deleted from archives!), but rather on the packages depending on the NBS package, to make those packages not depend anymore on the zombie.
(04:12:43 PM) warp10: In fact, our final goal is to make sure that the NBS package has no reverse-dependencies (i.e.: no packages depending on it), making it a dead leaf we can drop with no pain from the archive.
(04:13:00 PM) warp10: A few tools are here to help you NBS, guys. The most important one is this page: http://people.canonical.com/~ubuntu-archive/NBS/
(04:13:31 PM) warp10: The full list of NBS waiting for love is there. It is updated about every 6 hours. As you see, every NBS is represented by a text file (named after the package name).
(04:13:59 PM) warp10: Let's open a NBS and see what's written there. For example, let's take libgss1: a pretty easy one. Straight link is: http://people.canonical.com/~ubuntu-archive/NBS/libgss1
(04:14:56 PM) warp10: This file contains informations about libgss1 reverse dependencies. In our case, two packages depend on libgss1 both on ia64 and sparc: libgss-dpg and libgss-dev. These two packages have to be rebuilt to drop the NBS.
(04:15:34 PM) warp10: You might be wondering why a simple rebuild is enough to fix everything. This happens thanks to the "{shlibs:Depends}" macro in debian/control who will link our package to the latest libraries available. In this case, it will drop the dependency on libgss1 and will link our packages to libgss3 (the brand new package who made libgss1 a NBS)
(04:16:05 PM) warp10: Let's see another example. Please, open libopensync0. As you see, this NBS has several reverse-dependencies that need to be updated. Anyway, don't despair. Very often, it's just a matter of rebuilding a few source packages that build several binary packages.
(04:16:46 PM) warp10: Let's go deeper in the matter with a third example. Please, open libv8-2.2.7: http://people.canonical.com/~ubuntu-archive/NBS/libv8-2.2.7.
(04:17:12 PM) warp10: As you see, there is just one package depending on libv8-2.2.7: it is nodejs on amd64, i386, and armel. Since (almost) everybody has either an i386 or amd64 machine, we can work on it.
(04:17:41 PM) warp10: Let's get some informations about nodejs. Let's download it from the archive and run dpkg -I on it: wget http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs_0.1.97-1_i386.deb && dpkg -I nodejs_0.1.97-1_i386.deb | grep Depends
(04:18:06 PM) warp10: As you see, it depends on libv8-2.2.7 (but we already knew that). Right now, the archive has libv8-2.2.18 and we should rebuild nodejs against it
(04:18:37 PM) warp10: Therefore, let's apt-get source nodejs. We will not make anything special about it, except for updating the changelog. Actually, we won't add the standard "ubuntu1" after the debian revision, but will rather use "build1" to show that we didn't modified anything and this is just a rebuild.
(04:19:05 PM) warp10: Anyway, given that we are asking the build daemons to rebuild the whole package, this is a good occasion to do other maintainance around nodejs. For example, we could check if there are open bugs, if the package is lintian free or if it needs to be merged/synced from Debian.
(04:19:34 PM) warp10: In any case, once we are over with the modifications we need (if any), we have to rebuild the package with our favourite tool (e.g.: pbuilder) and after the standard Build-Install-Run test, we can upload it to archives, or ask a sponsor to do it for us.
(04:20:14 PM) warp10: Fixing a NBS isn't always that easy. Sometimes your package FTBFS, or there are other issues around that will make your life not easy. Don't forget that your fellow Ubuntu Developers are around and ready to help and assist you
(04:20:51 PM) warp10: Ok, everything clear so far? Is your mind all messed up with reverse-deps and NBS? :)
(04:21:19 PM) warp10: A final word about another tool the Good Old Gaspa made available for all of us. Please, head your browser to http://gaspa.yattaweb.it/issues/reverse_nbs.xml at maximum warp speed.
(04:21:53 PM) warp10: As you can easily understand from the name itself, this page allows you to tackle NBSs from the build-deps side. Rather than opening the NBS page and reading what package needs to be rebuilt, you here see what package depends on which NBSs (yes, even multiple ones)
(04:22:33 PM) warp10: This way, you can check the status of a particular package you are working on, and can easily understand if that package needs multiple transitions.
(04:23:11 PM) warp10: That page also shows some other interesting information, like the already done NBS (i.e.: ready to be dropped from archives), and lots of links to launchpad, packages.u.c, the NBS page on people.u.c, all with an eye-candy UI (altough gaspa should update it to the lucid's aubergine colour :))
(04:23:39 PM) warp10: Summarizing, NBSs are a nice, funny, and appreciated QA activities (we can't release Ubuntu if NBSs are around in the archives!) and you have at least two tools and 100+ Ubuntu Developers ready to help you.
(04:24:10 PM) warp10: Therefore, choose your favourite NBS and kill him rebuilding all of its reverse-deps! :)
(04:24:54 PM) warp10: And now, let's move to a completely different topic: FTBFS. Unless you have questions to ask in -chat, of course.
(04:25:36 PM) gaspa: :)
(04:25:52 PM) gaspa: First of all, presetation: hi everybody! I'm gaspa, MOTU, a python passionate (well, passionate of
(04:25:52 PM) gaspa: thousands of other things, but that's what matters),
(04:26:33 PM) gaspa: and then,let's talk about QA: as you may know, we have a lot of packages in ubuntu but not all of them were successfully built.
(04:26:55 PM) gaspa: When this happens we say that this package "Fails To Build From Source", but usually we use the acronym FTBFS for indicate that.
(04:27:32 PM) gaspa: A package can FTBFS for a lot of reasons, the most common reason is for its dependencies... for example some dependencies could have a different version than debian.
(04:28:08 PM) gaspa: what I said above is important: in fact if a package FTBFS in ubuntu it does not mean that it will FTBFS in debian...
(04:28:26 PM) gaspa: ( so we should often figth them with our forces ;) )
(04:29:19 PM) gaspa: but, of course, we have our tools that gives help: guess what? we havea list of packages that FTBFS, see http://qa.ubuntuwire.org/ftbfs
(04:30:33 PM) gaspa: the page lists all the packages, together with a lot of helpful informations, the arch that fails, links to LP and packages.debian.org pages....
(04:31:04 PM) gaspa: but the most important is the build log, that help us understand really what gone wrong!
(04:31:27 PM) gaspa: In the best case you just need to do a fresh build of it, or perhaps to update a single dependency...
(04:31:51 PM) gaspa: but other issues can happen too, of course...
(04:32:27 PM) gaspa: for example, another reason is that we have sometimes different compiling option from debian or upstream, for this kind of problems google helps greatly (search for the string that gives the error...)
(04:32:58 PM) gaspa: i.e. it happened with different versions of glibc between debian/ubuntu
(04:35:01 PM) gaspa: of course we have a lot of programming languages, every kind of language has different behavior and different problems... You can concentrate on your preferred language and find the package of that language.
(04:35:19 PM) gaspa: Last, but not the least, it can be an upstream bug: remember always to report them (together with the fix you found, of course), so the package will be updated in debian and then we can sync/merge it
(04:35:58 PM) gaspa: now, another kind of QA we can do... merges!
(04:36:18 PM) ***gaspa throw the mike toward warp10
(04:37:03 PM) ***warp10 grabs the microphone and reaches the center of the classroom
(04:37:50 PM) warp10: Ok, we all know Ubuntu is a debian-derived distro. This means we have the same packages too. We have a few procedures to update our packages from debian to ubuntu. We can do a sync if the package has no ubuntu changes, otherwise we need to merge all ubuntu changes for the new package from debian.
(04:38:22 PM) warp10: First of all we have to check if our ubuntu changes have been accepted in the debian package. We can sync the package only if *every* change in the ubuntu package has been accepted in the debian package too
(04:38:53 PM) warp10: Why do we need to do those changes? Well, for example we want to fix a bug in our ubuntu package which is not fixed yet in Debian, or we need to do ubuntu-*specific* changes.
(04:39:24 PM) warp10: Let's go ahead with merges. How can we merge a package? First of all we need to check the pending merges with MoM, a web application hosted on merges.ubuntu.com
(04:39:52 PM) warp10: Once we are there we will see the merges sorted from 4 components (universe, multiverse, main and restricted). In this session we will work on the pending merges of the universe component, so let's visit merges.ubuntu.com/universe.html
(04:40:27 PM) warp10: And please, don't forget to contact the latest uploader of the package (to prevent doubling the work).
(04:40:43 PM) warp10: You can do a merge in many ways: either manually, or with the grab-merge script. In this session we will use this second method.
(04:41:05 PM) warp10: Where can we download the grab-merge script? It has been included in the ubuntu-dev-tools package, so just install the ubuntu-dev-tools package.
(04:41:25 PM) warp10: Let's suppose we will work on the 'vlc' package, so type 'grab-merge vlc' and read the REPORT file for useful informations.
(04:41:43 PM) warp10: We will check the changelog of ubuntu and debian (let's suppose we have the version 1.0.5-1ubuntu1 and the version 1.0.6-1 in debian).
(04:42:00 PM) warp10: Imagine we have the ubuntu change * Add foobar to BD (LP: #123456). First of all, we will edit our debian/changelog file (changing the description of the change and replacing the mom signature with our own).
(04:42:23 PM) warp10: Then we will merge that change adding the BD foobar in debian/control
(04:42:38 PM) warp10: If you're unsure, don't forget you can visit patches.ubuntu.com and check with the latest debdiff (ubuntu+version.patch).
(04:42:59 PM) warp10: Once you are done you have to run ../merge-genchanges in the new, merged package, and then ../merge-buildpackage to check the package builds correctly.
(04:43:26 PM) warp10: If the package builds fine, you have to open a bug report in LP against the package with the summary: "Please merge vlc 1.0.6-1 (universe) from debian unstable (main) (if it's in debian unstable in the component 'main', of course)
(04:44:05 PM) warp10: Now, you can attach your debdiff to the bug report. Otherwise, you can use bzr to create a branch with your debdiff.
(04:44:21 PM) warp10: You're done! just subscribe ubuntu-sponsors and wait patiently. :)
(04:44:38 PM) warp10: As I said above, we can merge with bzr too. Let's see how that works.
(04:45:00 PM) warp10: First of all, we need to see if the package we want to merge has any failure http://package-import.ubuntu.com/status/
(04:45:15 PM) warp10: Let's suppose we want to merge vlc from debian sid again. Run: bzr merge-package lp:debian/sid/vlc
(04:45:38 PM) warp10: Then we need to run 'bzr status' and 'bzr diff' to check if there are conflicts or something else wrong
(04:45:58 PM) warp10: For example, if there are conflicts, you have to run bzr resolve and bzr conflicts, and  then you can do an entry in debian/changelog with the command dch -i
(04:46:28 PM) warp10: Now our branch is ready to be committed, but run bzr diff -r tag:1.0.6-1 first to check all ubuntu changes. You're done!
(04:47:06 PM) warp10: This concludes the part of this session about merges. The last quarter will be held from Good Old Gaspa who will talk about ubuntuwire
(04:47:15 PM) ***warp10 throws the microphone back to gaspa
(04:47:18 PM) ***gaspa 's not Old
(04:47:33 PM) ***gaspa catch the mike with a double-back-flip
(04:47:43 PM) ***warp10 but he his Good! \o/
(04:47:52 PM) gaspa: ok, warp10 already talked about http://qa.ubuntu.com....
(04:47:55 PM) gaspa: lol
(04:48:24 PM) gaspa: another resources I want to point all fo you is UbuntuWire... Ubuntuwire is a
(04:48:24 PM) gaspa: website that collect a lot of services that fit into the ubuntu community: lists, search, and a lot of QA resources.
(04:48:55 PM) gaspa: point your browsers to http://ubuntuwire.com
(04:49:30 PM) gaspa: Unfortunately there's no time to look in deep at each of them, but a look on the page above will give you a bit of taste.
(04:49:35 PM) gaspa: There are community driven resources, as well as Canonical ones. For example you already saw NBS (the first of Canonical resources), and the ftbfs page.
(04:49:50 PM) gaspa: Well, you'd already have a lot of material to look at, tonight ;)
(04:50:00 PM) gaspa: isn't it? :)
(04:50:08 PM) gaspa: Although, but I want to point you on a couple of these pages that I found particularly useful.
(04:50:42 PM) gaspa:  the first of them is the debcheck page: http://qa.ubuntuwire.org/debcheck/ This page lists all the package that aren't in good shape from the point of view of their relationships (packages that feel alone, poor packages :) )
(04:51:07 PM) ***warp10 chuckles
(04:51:16 PM) gaspa: If you give a fast look at that page, you'll see a lot of classes of issues: Broken Depends/Recommends/Suggests, Build-Depends, HalfBroken, Packages in Main with depends on !Main...AWESOME, we'll have a lot of work!! :)
(04:52:02 PM) gaspa: As you can see, pages are grouped by issue, but if you select a single package, you'll see a lot of issues for only one! So, you can make the package shine resolving a lot of problems. [well, to be honest, often a lot of issues are caused by a single problem, so it's probably easier to fix them in a while)
(04:52:27 PM) gaspa: example! let's take a look at a single package:
(04:52:35 PM) gaspa: http://qa.ubuntuwire.org/debcheck/debcheck.py?dist=maverick&package=anjal
(04:53:03 PM) gaspa: you can see that "Package declares a build time dependency on evolution-plugins (<< 2.29.0) which cannot be satisfied on ia64. evolution-plugins (<< 2.29.0) 2.30.1.2-3ubuntu1 is available." So, anjal depends on evolution-plugins << 2.29.0, but we have 2.30 in maverick!
(04:53:07 PM) gaspa: What should we do!?
(04:53:49 PM) gaspa: Of course this issue could be caused by some different reasons. One could be simply a build-dependency that need to be updated.... is this the case? [ Hint: why there's a '<<' on the build-dependency and not a "<=" ?? ]
(04:54:15 PM) gaspa: oh, another possibility is that anjal depends on a feature that's present on evolution-plugins in 2.29, but is moved on another package from 2.29 and so on...
(04:54:48 PM) gaspa: note that *I don't know the real answer*, so perhaps it's worth a rebuild on your ppa and test the package with only a version dump.
(04:55:39 PM) gaspa: we can look at how behave the build in your ppa, and let's proceed in the right direction
(04:55:45 PM) gaspa: thanks, ClassBot.
(04:56:11 PM) gaspa: Little hint: If you have doubts, well, you can always take a look at Debian packages... they should be always your reference [in particular seems this package does not have a debian maintainer... any volunteer in here? :) ]
(04:56:17 PM) gaspa: :)
(04:56:48 PM) gaspa: Another thing that you should remember (haven't said it yet??) is to keep in contact with upstream, when possible...
(04:56:53 PM) gaspa: asking upstream is perhaps a bit longer than resolve problems ourselves, but have the advantage that tipically upstream knows better the software in case :)
(04:57:15 PM) gaspa: Ok, we met debcheck. Keep in mind that you can run itself on your PCs,it's enough to install edos-distcheck and make it runs :)
(04:57:32 PM) gaspa: for example, I did on my own and I publish sometimes the results:
(04:57:41 PM) gaspa: http://gaspa.yattaweb.it/issues/issues/edos/lucid_i386_edosresults.xml
(04:58:01 PM) gaspa: ( i didn't do any run for maverick, yet... :P I'll do one soon, I promise! )
(04:58:27 PM) gaspa: ok, I wanted to cite Harvest, but there's no time, I guess...
(04:59:03 PM) warp10: gaspa: I suppose we can steal a few more minutes, if needed
(04:59:04 PM) gaspa: so, just some time for question...
(04:59:28 PM) gaspa: warp10: if ClassBot wont shut up us :)
(04:59:39 PM) warp10: gaspa: don't think so. He is such a good boy :)
(04:59:48 PM) gaspa: ok :)
(04:59:53 PM) gaspa: let's try ;)
(05:00:19 PM) gaspa: I'll be brief: the approach we got  till now, is something like "take a kind of bug and fix as many package as you can"
(05:00:24 PM) gaspa: merges, ftbfs, nb...
(05:00:35 PM) gaspa: This is handful sometimes, expecially when you want to became experienced on something in particular
(05:00:57 PM) gaspa: But, have you noted that every of these pages/tools we showed so far differs, is in a different place from the others? well, wouldn't be more efficient to take a package and fix all the problem of that package? So, for this kind of QA,dholbach build Harvest
(05:01:15 PM) gaspa:  Harvest's aim is to centralize a lot of these lists of bugs, and show them in a comfortable way, and makes us able to fix a lot of bugs of a single package without searching here and there in the internet :) ... so, it makes us *really* happy :)
(05:01:43 PM) gaspa: I can't show you as daniel is working hard with a GSoC student to get it back more cool, usable, extensible, and whatever :P... but STAY TUNED on the link at the qa.ubuntu.com page :)
(05:02:18 PM) gaspa: ok, finished. ;) thanks and sorry for taking a bit of extra time :)

MeetingLogs/devweek1007/QA (last edited 2010-07-16 21:14:10 by gaspa)