I, Daniel Manrique, apply for upload rights for package checkbox.
Who I am
A long-time Linux user, I started using Ubuntu in 2009 and landed a job working at Canonical in 2010. I'm part of Canonical's hardware certification team, so most of my experience regarding Ubuntu development has been with Checkbox, the tool we use to run certification tests. Checkbox is also shipped with Ubuntu and is used as the client for the Ubuntu Friendly community testing program. As part of Ubuntu, checkbox receives a fair share of community bug reports, which I help triage, and I usually am responsible for preparing versions of Checkbox to be uploaded to Ubuntu via a sponsor. I've also contributed bits and pieces to other projects, mainly for bugs we've run across when debugging problems we find while certifying.
My Ubuntu story
I've been using Ubuntu since 2009 (when I migrated away from Debian). I opened a Launchpad account pretty early on to report on some problems I was having. My efforts within Ubuntu have mostly focused on bug reporting and helping the OS developers with triaging by performing the tests they require.
As mentioned, I've also done work on the tools that the Hardware Certification team uses to test and certify hardware. Chief among these is the checkbox extensible testing tool. The improvements to Checkbox we constantly need are what drove me to get involved in its development. I take part in all aspects of its development cycle: filing and triaging bugs, identifying work to be done to fix them, coding and testing fixes, and managing the merging of code both to the main Checkbox trunk and to the Ubuntu branch.
Examples of my work / Things I'm proud of
Areas of work
Non-checkbox areas of work within Ubuntu include Ubiquity and Casper (that bug resurfaced here and Stéphane Graber was kind enough to squash it). The folks in charge of these projects are always very responsive and patient, which I really appreciate.
Checkbox's situation has changed in the past several months. The Hardware Certification team has been growing in terms of people who can devote time working on Checkbox. This has meant we're much more able to tackle the large bug backlog the project had, while also adding new features and keeping up to date with Ubuntu's evolution. Since a large part of Checkbox are the test scripts, and these need to interact with the kernel and other APIs that keep evolving, they need constant maintenance and updating. Examples that come to mind are the recent migration to UDisks2 (for Ubuntu Quantal), the upgrade to Python3 (Quantal as well), migration to Gtk3 (for Ubuntu 11.10), and changes to APIs for Network Manager.
As a team we've managed to greatly reduce the amount of open and new bugs for https://bugs.launchpad.net/ubuntu/+source/checkbox/ (Currently 16 open bugs and 2 new ones). Checkbox's trunk/upstream branch still has a large number of open bugs (bug list here). However we've drastically reduced the number of new, unlooked-at ones, and I've been focusing on analyzing open bugs and giving them a clear "statement of work" for developers to work on; this way they can potentially jump straight into coding a solution. I felt this was more conducent to many people working on these triaged bugs, as opposed to i.e. me going over them one-by-one, analyzing, and fixing them. This way developers, both from our team and outside, can work in "parallel" to fix these bugs, and they also don't need to spare brainpower finding a bug to fix.
I've also been trying to tag smaller bugs with "bitesize" to encourage community participation. This has indeed resulted in a few of these bugs being fixed by community members. We've also tried to be extra responsive to contributions to Checkbox from outside the certification team, to further encourage contributions, this has also resulted in more people knowing about checkbox and potentially considering it when searching for a test runner application.
As a result the number of commits per release has grown dramatically, from 30 for the Natty cycle to 217 for the Oneiric cycle, 334 for Precise and 301 for Quantal. Particularly for the Precise cycle, this resulted in our uploads to Ubuntu having very large changelogs, which made reviewers' lives difficult. So for Quantal, despite having slightly less commits and bugfixes, we had a more regular upload cadence, with smaller changesets to ease work for reviewers and sponsors.
Also, in order to ensure our package uploads are sane, we've been ramping up our "unit test" coverage, which also validates that data files contain no errors, that translated versions don't cause crashes, and that the code passes some basic "lint" tests. These processes have already caught a few errors. We complement this with a daily from-trunk PPA build which sends a notification if these automated tests fail, so we can catch these basic problems even if some bogus code makes it into the trunk branch.
Things I could do better
- I need to follow process more carefully, particularly for exceptions (FFe and UIFe). I tend to leave an assessment on whether they are needed for the reviewer/sponsor, and so far it's been pretty safe to assume that, if there's any doubt about it, then an exception is probably needed.
- I need to keep better track of deadlines to avoid last-minute emergencies. Mathieu Trudel-Lapierre often sponsors checkbox uploads and he can attest to this. This is compounded by the earlier point about exceptions; sometimes I get uploads ready on time, but because we didn't ask for an exception, we end up being right on the deadline.
- I still need to step up on bug triaging for Checkbox. We've had a lot of progress but we still depend on "bug triaging binges" to slash the bug backlogs. I need to do this more regularly instead.
Plans for the future
We want to add even more automated testing to Checkbox, this will allow us to be more confident when uploading and should also reduce the amount of uploads we need to do. Once the bug backlogs are smaller we also want to see if we can improve our turnaround time for fixing bugs and releasing packages with the fixes.
What I like least in Ubuntu
Pertaining development, my experience is basically with Checkbox, and the few aspects where I think improvements could be made are:
- Testing, I think some of the problems we have encountered with checkbox uploads could and should have been caught by automated testing. We continue to evolve our testing suite since the results we've had with e.g. automated job syntax checking have been very positive, but this is done on an ad-hoc basis and I think it's something that the Ubuntu process should both suggest and maybe somehow enforce.
- Criteria unification. Sponsors do a great job of applying their better judgement when reviewing and sponsoring uploads, but when freezes hit, processes depend a lot on exception approval (understandable since at these points we want to reduce the risk introduced by features), and these exception approvals depend somewhat on criteria that leave some room for interpretation. So changes that may keep some sponsors and reviewers happy may be rejected by others. This has caused some confusion for the checkbox development team (and I imagine others have encountered the same), so some uniformity here would help make things clearer and easier to deal with. It's not an easy problem, though, as a process that's too rigid is like shackles and risks making everybody unhappy.
If you'd like to comment, but are not the applicant or a sponsor, do it here. Don't forget to sign with @SIG@.
As a sponsor, just copy the template below, fill it out and add it to this section.
== <SPONSORS NAME> == === General feedback === ## Please fill us in on your shared experience. (How many packages did you sponsor? How would you judge the quality? How would you describe the improvements? Do you trust the applicant?) === Specific Experiences of working together === ''Please add good examples of your work together, but also cases that could have handled better.'' === Areas of Improvement ===