Ubuntu Wiki

   1 [17:00] <jibel> Hi folks,
   2 === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Developer Week - Current Session: How Stable Release Updates work - Instructors: jibel
   3 [17:01] <ClassBot> Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/01/%23ubuntu-classroom.html following the conclusion of the session.
   4 [17:01] <jibel> Thanks to smspillaz for this awesome introduction to compiz.
   5 [17:01] <jibel> my name is Jean-Baptiste Lallement, I live in France and I'm working on the QA Team.
   6 [17:02] <jibel> I've started my QA activity with lot of bug triaging, then I moved to ISO Testing, Automated Desktop Testing and Stable Release Updates.
   7 [17:02] <jibel> In this session we will talk about Stable Release Updates, and how the QA process contributes to rock solid stable releases.
   8 [17:03] <jibel> Once an Ubuntu release is out, updates for it follow a very specific procedure called a "Stable Release Update" aka "SRU"
   9 [17:03] <jibel> Updates are only released under certain circumstances.
  10 [17:04] <jibel> Why this SRU process ?
  11 [17:05] <jibel> Unlike development releases (at the moment Natty), official releases of Ubuntu are widely deployed and used by a wide variety of users.
  12 [17:05] <jibel> During the development, the population are developers, early adopters or advanced users, they are aware that pre-release software may break and they accept the risk.
  13 [17:06] <jibel> When you are using an official release, you use your system for your day to day work and are expecting a high degree of stability.
  14 [17:06] <jibel> Any problem can be really disruptive and the system should not requires the intervention of the user.
  15 [17:08] <jibel> Stable release updates are automatically recommended to a very large number of users. So it is critically important to be paranoïd and to treat them with extreme cuation.
  16 [17:08] <jibel> Therefore, when updates are proposed, they must be accompanied by a strong rationale and present a low risk of regressions.
  17 [17:09] <jibel> Some will argue "It's just a one line change, go ahead"
  18 [17:09] <jibel> Right, let me give you one example, with bug 559822 and its friend bug 610975
  19 [17:10] <jibel> The cause of the bug wasa C++ library (wxwidgets2.8) was uploaded with no code changes. Due to an underlying toolchain change/bug, this caused an ABI change, causing a lot of unrelated packages to break.
  20 [17:10] <jibel> It was not a one line change, it was a no change at all bug.
  21 [17:11] <jibel> So people, when dealing with SRU, expect the unexpected.
  22 [17:12] <jibel> When should you request a Stable Release Update ?
  23 === dholbach_ is now known as dholbach
  24 [17:12] <jibel> Not all bugs are good candidates to Stable release updates,
  25 [17:13] <jibel> and in general only those that fix high impact bugs are
  26 [17:13] <jibel> examples of such bugs include:
  27 [17:14] <jibel> * security vulnerabilities. These are done by the security team and follow a slightly different but somewhat similar process
  28 [17:14] <jibel> * severe regressions from a previous release.
  29 [17:15] <jibel> This includes packages which are unusable, like being uninstallable or crashing on startup.
  30 [17:15] <jibel> * bugs which may cause data loss
  31 [17:15] <jibel> bugs which doesn't fit under the above categories but have an obviously safe patch
  32 [17:16] <jibel> and affect an application rather than critical infrastructure packages (like X or the Kernel)
  33 [17:17] <jibel> * for Long Term Support (LTS) we regularly want to enable new hardware.
  34 [17:17] <jibel> such changes are appropriate provided that we can ensure to not affect upgrades on existing hardware
  35 [17:17] <jibel> And the last type of SRU
  36 [17:18] <jibel> * New version of commercial software in the canonical partner archive
  37 [17:18] <ClassBot> wolfpack asked: Is stable release update has only bug fixes or can it have features upgrade ?
  38 [17:19] <jibel> SRU doesn't introduce new feature, or with rare exceptions.
  39 [17:20] <jibel> For new features, or new upstream versions of packages, but don't fix critical bugs you should request a backport instead.
  40 [17:20] <jibel> New features are usually not good candidates.
  41 [17:22] <jibel> So How to proceed ?
  42 [17:22] <jibel> You've found a good candidate ? There are a few steps that you must follow in order to see your fix land in a Stable Release.
  43 [17:23] <jibel> First of all, the bug must be fixed in the current development release
  44 [17:23] <jibel> and the task in the bug report set to "Fix Released"
  45 [17:24] <jibel> This ensures that the fix won't be missed on newer version of the package and that the bug won't be reintroduced
  46 [17:24] <jibel> If it is not fixed, in the development version, the request for SRU will be rejected.
  47 [17:26] <ClassBot> darkdevil58 asked: if suppose i fix a bug on my 10.04 system, still the fix release be rejected as i m not using 10.10?
  48 [17:28] <jibel> If you want a fix in 10.04 LTS, it must be fixed in the latest development release - Natty - before being ported to 10.04
  49 [17:28] <jibel> We can accept an SRU for 10.04 but which is not necessarily a good candidate for 10.10
  50 [17:29] <jibel> When filing an SRU add all necessary the information to the bug description to help with the review of the report. Ideally, make sure it contains:
  51 [17:29] <jibel> * The rationale for the SRU. the impact of the bug on users, and a justification for backporting the fix to the stable release
  52 [17:30] <jibel> * Explain how the bug has been addressed in the dev branch, and attach a minimal patch against the current stable version of the package.
  53 [17:30] <jibel> At this point, if you think that preparing the patch will be time consuming,
  54 [17:30] <jibel> you should get approval from the SRU Team first.
  55 [17:31] <jibel> On the other hand its useless to spend time on a patch that is not critical enough to match SRU criteria.
  56 [17:31] <jibel> So, in doubt, don't hesitate to ping the SRU team
  57 [17:32] <jibel> An important part, document the test case with detailed instructions how to reproduce the bug.
  58 [17:33] <jibel> Document the test case with detailed instruction how to reproduce the bug
  59 [17:33] <jibel> for example https://bugs.launchpad.net/ubuntu/+source/software-center/+bug/639701
  60 [17:33] <jibel> includes 3 easy steps to reproduce the issue, and verify the fix.
  61 [17:34] <jibel> The steps should allow anyone, not familiar with the package to reproduce the bug and verify that the issue is fixed with the updated package.
  62 [17:34] <jibel> * Add information about potential regressions and inadvertent side effects.
  63 [17:35] <jibel> For example in https://bugs.launchpad.net/ubuntu/+source/txaws/+bug/634022 , the dev explains why there's very little risk of regression
  64 [17:35] <jibel> Once you've provided all the required documentation
  65 [17:35] <jibel> * Target to the appropriate serie and subscribe ubuntu-sru
  66 [17:36] <jibel> * Upload the fixed package. If you can not do it yourself, attach a debdiff and subscribe ubuntu-sponsor.
  67 [17:36] <jibel> * If everything goes right, the archive admins will approve and publish you fixed package to the -proposed repository.
  68 [17:36] <jibel> And please, keep this rule in mind: One bug = One report
  69 === drubin_ is now known as drubin
  70 [17:36] <jibel> Then we enter in the second phase of the Stable Release Update: SRU Verification
  71 [17:36] <jibel> The current set of pending SRUs is tracked at http://people.ubuntu.com/~ubuntu-archive/pending-sru.html
  72 [17:37] <jibel> On this page, you'll find a list of packages currently in -proposed
  73 [17:37] <jibel> and available for testing.
  74 [17:38] <jibel> For each package, there's an indication of which version is in which pocket (-security, -updates, -proposed) and a list of bugs fixed by the upload
  75 [17:38] <jibel> The process we follow to verify a package is roughly:
  76 [17:38] <jibel> - setup a testing environment
  77 [17:39] <jibel> - Try to reproduce the issue in the most recent published version of the software
  78 [17:39] <jibel> - Enable the -proposed repository and install the proposed update for that software
  79 [17:39] <jibel> - Try to reproduce the issue with the updated software
  80 [17:40] <jibel> The first step is to create a clean testing environment of the release the fix is targeted for.
  81 [17:40] <jibel> We want to eliminate any outside effects and control the changes brought by the update of the package.
  82 [17:41] <jibel> To create this environment any virtualization solution is good: KVM, VirtualBox, VMWare, ... are very useful to create an isolated testing environment
  83 [17:41] <jibel> You can use snamshot to quickly setup a fresh env or rollback to a previous state.
  84 [17:42] <jibel> Doing testing in a chroot is also good enough.
  85 [17:42] <jibel> If you are trying to reproduce hardware related issues, then, of course, VM technologies are not the right choice.
  86 [17:42] <jibel> Hardware specific bugs offer their own special problems; if we don't have the hardware, it's much harder to reproduce
  87 [17:42] <jibel> There's not much we can do except prompt on the bug report for people affected by the bug to do the testing and install the package ourselves to look for regressions.
  88 [17:43] <jibel> These are often kernel, X, libc (where the compiler is bounded to CPU capabilities) problems
  89 [17:43] <jibel> Then we want to make sure that all updates are applied to the testing environment and we can start the testing and try to reproduce the bug
  90 [17:43] <jibel> We want to do this so that we know what we're testing for when we install the proposed package.
  91 [17:44] <jibel> Often, just doing this will offer us more insights into the actual problem being solved and sometimes discover a regression or another bug.
  92 [17:44] <jibel> If we can't reproduce the initial problem, we have much less confidence that the update actually is going to fix the issue.
  93 [17:45] <jibel> In this situation, whether the Test Case is wrong or incomplete, or the condition to reproduce the bug are not well defined, so we can have a doubt that the fix will really fix the bug in every conditions.
  94 [17:47] <jibel> l'll illustrate the verification phase with bug https://bugs.launchpad.net/ubuntu/+source/metacity/+bug/494096
  95 [17:47] <jibel> This is a critical usability defect. Clicking on a window send the events to the underneath window.
  96 [17:47] <jibel> wrt to this bug a patch has been attached in comment #23 and the bug has been nominated for SRU between comments 25 and 26.
  97 [17:48] <jibel> The description is fairly complete, the issue is easy to understand and it includes a test case.
  98 [17:48] <jibel> That's not always the case, so sometimes you get to figure out how to test for the condition, or if that's really unclear what the bug is and how to reproduce, you can simply set the status back to 'in progress' asking for more info from the reporter or the dev.
  99 [17:48] <jibel> The first task is to reproduce the issue.
 100 [17:49] <jibel> Just follow the steps described in the bug report. Once this is done you can enable -proposed
 101 [17:49] <jibel> You can do this through the menus, via System -> Admin -> Software Sources menu
 102 [17:49] <jibel> The details to enable -proposed are at https://wiki.ubuntu.com/Testing/EnableProposed
 103 [17:50] <jibel> As part of reproducing, make sure to double-check exactly which version you are testing
 104 [17:50] <jibel> From the command line, apt-cache policy is your friend, and in this case
 105 [17:50] <jibel> $ apt-cache policy metacity
 106 [17:50] <jibel> metacity:
 107 [17:50] <jibel>   Installed: 1:2.30.1-0ubuntu1
 108 [17:50] <jibel>   Candidate: 1:2.30.1-0ubuntu1.1
 109 [17:51] <jibel>   Version table:
 110 [17:51] <jibel>      1:2.30.1-0ubuntu1.1 0
 111 [17:51] <ClassBot> There are 10 minutes remaining in the current session.
 112 [17:51] <jibel>         500 http://archive.ubuntu.com/ubuntu/ lucid-proposed/main Packages
 113 [17:51] <jibel>  *** 1:2.30.1-0ubuntu1 0
 114 [17:51] <jibel>         500 http://fr.archive.ubuntu.com/ubuntu/ lucid/main Packages
 115 [17:51] <jibel>         100 /var/lib/dpkg/status
 116 [17:51] <jibel> We see that 1:2.30.1-0ubuntu1 is installed and the candidate 1:2.30.1-0ubuntu1.1 from -proposed is available for update.
 117 [17:51] <jibel> You can install it either with update-manager or from the command line:
 118 [17:51] <jibel> $ apt-get install metacity
 119 [17:52] <jibel> Only update this package and its dependencies, but not all the packages from -proposed as they may introduce some other issues.
 120 [17:52] <jibel> Once it is installed logout/login and try to reproduce the bug again.
 121 [17:53] <jibel> If the result is what you expect from the test case, we can start looking for regressions as well.
 122 [17:53] <jibel> Some things to look for in term of regressions:
 123 [17:53] <jibel> * install requirements; does it depend on other packages in proposed, particularly ones not from the same source package?
 124 [17:54] <jibel> if it does, and both packages aren't moved into updates at the same time, the package will be uninstallable.
 125 [17:54] <jibel> * possible differences in behavior for fresh install versus update from buggy version
 126 [17:54] <jibel> sometimes the fixes only show up for new installations, particularly for packaging problems
 127 [17:54] <jibel> just generally play with the app; helps here if you're familiar with it.
 128 [17:54] <jibel> For metacity, that's pretty straightforward, we'll go ahead use our desktop, and see if things still work properly
 129 [17:54] <jibel> If everything passes then tag the report with verification-done
 130 [17:54] <jibel> Quicky another example where things didn't went so well, which affects ssh
 131 [17:55] <jibel> https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/713002
 132 [17:55] <jibel> ssh is a key component of  the system and it is critical to not break it.
 133 [17:55] <jibel> The bug is that you cannot disable access from IPv4 hosts to IPv6 only hosts. This makes life of sysadmins harder to keep sane access control where IPv6 is the primary protocol.
 134 [17:55] <ClassBot> There are 5 minutes remaining in the current session.
 135 [17:56] <jibel> You can see that the bug description has been rewritten by the developer to improce the documentation. He was kind enough to keep the history of the original report for reference.
 136 [17:56] <jibel> He also added a test case, every commenters and tester on the report agreed to say that the bug was fixed.
 137 [17:56] <jibel> But...
 138 [17:56] <jibel> A security guy comes in and said "STOP! this fix doesn't includes the fixes from previous version"
 139 [17:56] <jibel> This is what is called a regression. The report has been tagged as verification-failed and the package will be removed from proposed.
 140 [17:56] <ClassBot> akshatj asked: What if a bugfix fixes a security issue but introduces another type of bug that is not security related like a usablity bug(not sure if this is possible)
 141 [17:57] <jibel> If a fix introduces a regression it is rejected.
 142 [17:58] <jibel> I don't have bug in mind to illustrate your question though.
 143 [17:58] <jibel> Finally both failures and successes are important to report!
 144 [17:58] <jibel> also report anything out of the ordinary or unexpected you had to do in testing
 145 [17:59] <jibel> it's also helpful to update the description of the bug if elements are missing (eg. you created a TESTCASE)
 146 [17:59] <jibel> We could use help in doing verifications and looking for regressions
 147 [17:59] <jibel> There's an SRU verification team on launchpad at https://launchpad.net/~sru-verification which tracks the SRUs,
 148 [17:59] <jibel> And if, when doing all this, you run into problems, you can always solicit for help on #ubuntu-bugs and #ubuntu-testing, as well as asking directly on the bug report.
 149 [17:59] <jibel> Anyone can contribute by doing SRU verification.
 150 [18:00] <jibel> any question ?