Ubuntu Wiki

Dev Week -- QA Tools -- ara -- Tue, Jan 20

UTC-4(EST)

(12:07:12 PM) dholbach: next up is Ara Pulido who will talk about QA Tools!
(12:07:18 PM) dholbach: let's hear a round of applause! :)
(12:07:30 PM) Arc: *cheers*
(12:07:40 PM) Bodom: *applause*
(12:07:44 PM) kranichTheII: *applause*
(12:07:47 PM) jpds: \o/ ara \o/
(12:07:50 PM) B_a_l_a_k: *applause*
(12:08:03 PM) theseas: *applause*
(12:08:06 PM) porthose: *applause*
(12:08:09 PM) ara: ok, enough, I am not Madonna
(12:08:29 PM) ara: Thanks all for coming and the warm welcome ;-)
(12:09:06 PM) ara: Questions in #ubuntu-classroom-chat, please
(12:09:19 PM) ara: My name is Ara and I am part of the Ubuntu QA Team (https://wiki.ubuntu.com/QATeam/)
(12:09:38 PM) ara:  Do you guys know what are the duties of the QA team?
(12:09:50 PM) kranichTheII: Quality Assurance ?
(12:10:11 PM) Arc: making sure the devs don't seriously break something for every Ubuntu user with an update
(12:10:58 PM) ara: yes, yes, it is Quality Assurance, and yes we have to try to make sure that devs don't break many things (something always break...)
(12:11:21 PM) ara: But the two most important duties are Triaging (https://wiki.ubuntu.com/BugSquad) and Testing (https://wiki.ubuntu.com/Testing).
(12:12:18 PM) ara: triaging bugs is about making bugs report better, or ask for more information, or provide better information, etc. so the devs can start working on them
(12:12:53 PM) ara: and testing gives a try to Ubuntu before it reaches the wider public
(12:13:17 PM) ara: both the buqsquad team, and the testing team are open to new contributors ;-)
(12:13:52 PM) ara: To perform our daily activities we have a set of tools to make the process easier and automate some of these tasks.
(12:14:41 PM) intellectronica left the room.
(12:14:52 PM) ara: To be able to try some of the tools that I am going to present, you will need to have installed 'bzr' in your computers. If you don't have it, you can install it easly:
(12:15:01 PM) ara: $ sudo apt-get install bzr
(12:15:15 PM) ara: all set?
(12:15:24 PM) kranichTheII: jup
(12:15:40 PM) ara: Let's start grabbing the LP project (https://code.edge.launchpad.net/ubuntu-qa-tools) that contains several QA tools.
(12:16:52 PM) ara: shankhs: complains that bzr doesnt support proxy and aythentication ... :(
(12:17:11 PM) ara: shankhs: I am not a bzr expert, but I am sure there must be some workaround there
(12:17:29 PM) ara: The trunk is the branch that it is better maintained, get the source using bzr:
(12:17:35 PM) ara: $ bzr branch lp:ubuntu-qa-tools
(12:18:11 PM) ara: The first set of scripts that we are going to visit are those that parse the bug mailing list. Those scripts are available in the newly created folder ./ubuntu-qa-tools/bug-mailinglist/
(12:18:43 PM) ara: all of these scripts are python scripts. be sure to have python on your machines
(12:19:13 PM) ara: The mailing list that receive every bugmail generated in Ubuntu has a copy at http://people.ubuntu.com/~listarchive/ubuntu-bugs/
(12:20:01 PM) ara: To use the bug-mailinglist scripts you have to grab a copy of one of the available archives and save it locally.
(12:20:20 PM) ara: this files are too big, for testing purposes you can just trim one of them
(12:20:52 PM) ara: The first script, triager-query.py queries the mailing list archive for actions performed by a specific bug triager.
(12:21:20 PM) kranichTheII left the room (quit: "Verlassend").
(12:21:28 PM) ara: actions like "add tag", "set as incomplete", "set as triaged", etc
(12:21:45 PM) ara: syntax go as
(12:21:46 PM) ara: python triager-query.py [email] [file]
(12:22:03 PM) ara: example:
(12:22:03 PM) ara: python triager-query.py sinzui.is@verizon.net ../test_bugs_mailing_list.txt
(12:22:03 PM) ara: From sinzui.is@verizon.net: 1
(12:22:03 PM) ara: sinzui.is@verizon.net triaged: 1
(12:22:27 PM) ara: this is the output of a trim of the January archive
(12:22:38 PM) ara: why this information is useful?
(12:22:54 PM) ara: As you might know, triagers in Ubuntu are divided in two teams. The bug-squad team is open to anyone, but to enter the bug-control team you have to be an experienced triager
(12:23:23 PM) ara: because bug-control members have access to potentially sensitive data
(12:23:56 PM) ara: If you're part of the bug-control team, one of your duties is to review new membership applications to the team
(12:24:47 PM) ara: count-senders.py parses the mailing list archive and returns the quantity of messages sent by individuals. This includes any activity in the bugs.
(12:24:48 PM) ara: python count-senders.py [email] [file]
(12:24:48 PM) ara: body-searching.py searches the mailing list archive for a user defined string, sys.argv[1], and returns bug numbers containing that string.
(12:24:48 PM) ara: python body-searching.py [string] [file]
(12:25:10 PM) ara: tagged-bugs.py searches the mailing list for tags, and creates a HTML report with the date the tag was added to the bug
(12:25:11 PM) ara: python tagged-bugs.py [tag] [file]
(12:25:50 PM) ara: all this information can be retrieve directly through Launchpad, of course, but if you have to do it daily, it is much easier to use the scripts
(12:25:54 PM) ara: and faster
(12:26:26 PM) ara: not saying that the launchpad web interface is slow... (in case some of the LP is watching...
(12:27:15 PM) furicle: ls
(12:27:35 PM) ara: any questions about the bugs-mailinglist scripts? If not I am going to move to a script to help testers
(12:28:52 PM) ara: bullgard4:  QUESTION: ':~$ python triager-query.py; python: can't open file 'triager-query.py': [Errno 2] No such file or directory'. What went wrong?
(12:29:31 PM) ara: bullgard4: did you move to the right folder? once you branch you have to move to ubuntu-qa-tools/bugs-mailinglist
(12:30:30 PM) ara: Let's move now to another tool, the dl-ubuntu-test-iso, a script that will help you keeping your testing images ISO up to date.
(12:30:46 PM) ara: have you ever helped with ISO images testing?
(12:32:07 PM) ara: then start now! ;-) https://wiki.ubuntu.com/Testing/ISO/
(12:32:44 PM) ara: each milestone released (alpha-1, alpha-2, point releases, etc.) need some ISO verification
(12:33:03 PM) ara: to avoid having to download every single iso each time you can have them synced
(12:33:26 PM) ara: dl-ubuntu-test-iso is based on rsync and can be configured to download only the necessary images.
(12:34:10 PM) ara: shankhs: and it is not a python script. it is bash scripting ;-)
(12:34:28 PM) ara: To configure the tool you need to save your configuration file as ~/iso/iso.cfg. You can see an example here: http://people.ubuntu.com/~ara/udw/qatools/iso.cfg
(12:35:08 PM) ara: In the example configuration file we have set to download jaunty release, only for i386 architectures, and alternate, desktop and server.
(12:35:08 PM) ara: Once you have set up the configuration file, and running ./dl-ubuntu-test-iso, the images will start to be download to the ~/iso folder.
(12:36:20 PM) ara: Please, take into account that by default all the images are download, so if you do not have a configuration file, you will download every possible image.
(12:37:04 PM) ara: If you run the script daily (or weekly), you will need to download only the changes between both images
(12:37:08 PM) ara: saving you time
(12:37:24 PM) ara: and bandwidth for your movies and music ;-)
(12:38:18 PM) ara: If you want to start helping with Ubuntu, ISO testing is a great way to start. and really, really helpful
(12:38:48 PM) ara: And now that we are talking about the dl-ubuntu-test-iso script to test ISO images, do you know the ISO tracker?
(12:39:40 PM) ara: The ISO tracker is a web application that we, at the QA team, use to track the testing of the ISO images. It is located at http://iso.qa.ubuntu.com/ and it is updated with the ISO images that we should be focusing in (right now the focus is taking by Hardy's point release).
(12:40:12 PM) ara: It is really helpful to create an account in the ISO tracker (if you have an account in the Brainstorm site, that one would work in the ISO tracker site as well) and submit your testing results.
(12:41:15 PM) ara: On the menu on the left, you can select the flavour that you're testing. Let's select Ubuntu flavour.
(12:41:52 PM) ara: Then you have to click on the ISO image that you're testing. Let's say that we are going to test Alternate i386 (the installer that does not have a Live session)
(12:42:08 PM) ara: When you click in one of the options, a list of testcases will appear, and a link to the ISO image, in case you haven't downloaded yet.
(12:42:39 PM) ara: Nice and easy
(12:43:05 PM) ara: In the list of testcases you can see what it is outstanding and have your pick taking that into account. Click on the chosen testcase to read it and submit your results.
(12:43:47 PM) ara: Any questions about the ISO testing site or ISO testing in general? (before I move back to bug triaging)
(12:44:53 PM) ara: bullgard4:  QUESTION: What is 'Hardy's point release'?
(12:45:14 PM) ara: bullgard4: Hardy (Ubuntu 8.04) is a LTS release (Long Term Support)
(12:46:59 PM) ara: bullgard4: we realease "point release" 8.04, 8.04.1, 8.04.2... that contain the latests updates already in the ISO images
(12:47:23 PM) ara: let's move to some useful reports that will make triagers life easier. When triaging bugs is useful to have access to some important information through the reports. Here's a list of some of them:
(12:47:49 PM) ara: Yesterday bugs: http://qa.ubuntu.com/reports/bugnumbers/yesterday.html
(12:47:49 PM) ara: Bugs reported yesterday. Useful to Invalid, Mark as Duplicate or request more information.
(12:48:19 PM) ara: With more duplicated bugs: http://qa.ubuntu.com/reports/launchpad-database/bugs-with-most-duplicates.html
(12:48:49 PM) ara: this is really useful when you're reading a bug that you "feel" "sure, this has been reported before"
(12:49:53 PM) tate_ is now known as teskew
(12:50:14 PM) ara: 25 oldest bug task that are <put here>:
(12:50:36 PM) ara: new: http://qa.ubuntu.com/reports/launchpad-database/oldest-new-ubuntu-bug-tasks.html
(12:50:36 PM) ara: Incomplete http://qa.ubuntu.com/reports/launchpad-database/oldest-incomplete-ubuntu-bug-tasks.html
(12:50:36 PM) ara: Confirmed http://qa.ubuntu.com/reports/launchpad-database/oldest-confirmed-ubuntu-bug-tasks.html
(12:50:36 PM) ara: Triaged http://qa.ubuntu.com/reports/launchpad-database/oldest-triaged-ubuntu-bug-tasks.html
(12:50:36 PM) ara: Progress http://qa.ubuntu.com/reports/launchpad-database/oldest-in-progress-ubuntu-bug-tasks.html
(12:50:38 PM) ara: Fix Committed http://qa.ubuntu.com/reports/launchpad-database/oldest-fix-committed-ubuntu-bug-tasks.html
(12:50:57 PM) ara: Helpful if you want to clean Launchpad from obsolete bugs
(12:51:21 PM) ara: The newest addition to the list, just announced a week ago, is the team assigned reports. As you might know, Canonical gives support to the packages in main, therefore, it is useful to have a list of the tasks assigned to one of this teams. The format of the URL of these reports is
(12:51:42 PM) ara: http://qa.ubuntu.com/reports/team-assigned/canonical-<team>-assigned-bug-tasks.html
(12:51:56 PM) ara: i.e. for the mobile team:
(12:51:56 PM) ara: http://qa.ubuntu.com/reports/team-assigned/canonical-mobile-assigned-bug-tasks.html
(12:52:21 PM) ara: A list of them can be found at http://qa.ubuntu.com/reports/team-assigned/
(12:52:42 PM) gman16k: http://qa.ubuntu.com/reports/team-assigned/
(12:53:20 PM) ara: These reports can be useful, in case a triager sees that an already assigned bug with high importance hasn't been touched in a while, he or she could ping the correct person directly, to try to push the resolution.
(12:54:02 PM) ara: triaging bugs involves a lot of talking through the IRC. If you like IRC, you will love triaging bugs!
(12:54:36 PM) ara: I am running out of time, I would like to move to testing again before I finish
(12:54:47 PM) ara: any questions on the triaging side?
(12:55:16 PM) ara: ok, let's move back to testing
(12:55:32 PM) ara:  we will talk a bit about SRU testing. Does anyone know what SRU stands for?
(12:55:48 PM) davmor2: stable release update
(12:56:16 PM) ara: SRU stands for Stable Release Updates. This is, those updates that are found in the Update Manager when we are running a stable release of Ubuntu.
(12:56:28 PM) ara: davmor2: you're a great student! ;-)
(12:56:58 PM) ara: Briefly, the process goes as:
(12:56:58 PM) ara: * An important bug, that needs to be fixed through updates, is found in a stable release.
(12:56:58 PM) ara: * A fixed is written and apply to the package. The package gets updated in the -proposed archive.
(12:56:58 PM) ara: * A tester willing to help with the SRU process (and has the -proposed repositories enabled; https://wiki.ubuntu.com/Testing/EnableProposed) installs the package and verifies that the bug has been fixed.
(12:56:58 PM) ara: * The fix is apply to the -updates repository.
(12:57:39 PM) ara: For more information about the SRU process you can go to https://wiki.ubuntu.com/StableReleaseUpdates
(12:57:54 PM) ara: To help tracking the outstanding bugs to test and help people coordinate the efforts, we have a very helpful tool, the SRU tracker:
(12:57:54 PM) ara: http://people.ubuntu.com/~sbeattie/sru_todo.html
(12:58:22 PM) ara: In this tracker you can easily spot what fixes need verification (and for which release) and a test case if available.
(12:58:46 PM) ara: If you're running a stable release of Ubuntu (hardy, intrepid...) this is a great way to help the QA team to make Ubuntu better.
(12:59:32 PM) ara: any questions before leonardr starts the following session?
(01:00:21 PM) ara: counting down:
(01:00:26 PM) ara: 5...
(01:00:28 PM) ara: 4...
(01:00:31 PM) ara: 3...
(01:00:33 PM) ara: 2...
(01:00:36 PM) ara: 1...
(01:00:39 PM) charlie-tca: Thanks ara, That was GREAT!
(01:00:48 PM) ara: Ok, that's it for today! I will be running another session on Thursday, to explain how to use Ubuntu Desktop Testing to create automated tests for the desktop.
(01:00:51 PM) davmor2: thanks ara
(01:00:54 PM) dinxter: cheers!
(01:00:57 PM) ara: Thanks all for coming!!