Ubuntu Wiki

Dev Week -- Kernel Triage -- JeremyFoshee -- Wed, Jul 14th, 2010

(04:00:58 PM) JFo: Hi folks, I'm JFo (Jeremy Foshee) and I am the Kernel Bug Triager
(04:01:18 PM) JFo: Today we are going to discuss a bit about triaging kernel package bugs.
(04:01:45 PM) JFo: this class is going to be something of a follow along for the chat from Saturday's Ubuntu User days
(04:01:58 PM) JFo: logs can be found here for that talk: https://wiki.ubuntu.com/UserDays/07102010/What%20is%20a%20kernel,%20and%20why%20do%20i%20need%20it
(04:02:11 PM) JFo: so several of the things I'd like to cover are:
(04:02:28 PM) JFo: 1) duplicate bugs and the Ubuntu Kernel
(04:02:45 PM) JFo: 2) The subsystem tagging of kernel bugs
(04:02:59 PM) JFo: 3) triage statuses and the kernel bugs
(04:03:10 PM) JFo: and 4) The Kernel Triage Summit
(04:03:29 PM) JFo: please feel free to ask questions as you see fit by prefacing them with QUESTION:
(04:03:41 PM) JFo: those of you using Lernid will not need to enter that
(04:03:48 PM) JFo: ok, duplicates :)
(04:04:27 PM) JFo: For those of you who may not be aware, the Ubuntu kernel team have decided not to continue using duplicates in the current way
(04:05:02 PM) JFo: this means that, where you would normally have marked bugs that were on the same laptop model, we'd now ask that you do not
(04:05:41 PM) JFo: this is due to several things, most notably, the fact that not all of the same laptop models carry the exact same chipsets
(04:06:02 PM) JFo: in some cases bus chips and minor board processors could be from different manufacturers etc.
(04:06:21 PM) JFo: we have found that minor differences such as these cause us issues when solving bugs
(04:06:38 PM) JFo: as we have many cases where a fix for one user did not solve other affected users
(04:07:16 PM) JFo: it was decided that we prefer for anyone affected by a bug to file a ticket for their own issue.
(04:07:37 PM) JFo: as such we have asked the launchpad team to provide us a way to turn off the ability to mark duplicates
(04:07:57 PM) JFo: and we are working on ways to preclude apport from recommending apparently related tickets
(04:08:46 PM) JFo: while this will add a heavy load to myself and the team, we feel it is imperative that we are able to work seemingly related issues differently
(04:09:05 PM) JFo: this will also allow us to get a better picture of who is affected by a bug and in what way
(04:09:22 PM) JFo: any questions on the change to the policy on dupes?
(04:09:43 PM) JFo: ok, moving on to item 2 :-)
(04:09:54 PM) JFo: subsystem tagging
(04:10:02 PM) ClassBot: penguin42 asked: OK, so what do you do when you think you have 50 related bugs - do you hae some common bug to attach them all to?
(04:10:39 PM) JFo: penguin42, no, in that case we would take a look at all of the bugs to identify underlying similarities
(04:10:58 PM) JFo: we'd then try to fix the underlying problem versus trying to fix symptoms
(04:11:27 PM) JFo: this would allow us to narrow our work to address the issue in a better manner.
(04:11:47 PM) JFo: It is also important to note that it is not the primary goal of the team to solve issues upstream
(04:12:11 PM) JFo: but this has the added benefit of helping us identify what upstream bugs could be related to a particular group of issues
(04:12:21 PM) ClassBot: simar asked: Regarding your duplicate policy, if we have duplicate bugs still the developers can have bugs grouped so they can work on like workable bugs in one go?
(04:12:48 PM) JFo: simar, we'd prefer not to use the duplicate feature in that way
(04:13:22 PM) JFo: in a perfect world, we could identify which bugs are actually duplicates and mark them such, but that takes a lot of overhead
(04:13:42 PM) JFo: in my perfect world, each of our bugs has an upstream bug watch :-)
(04:14:11 PM) JFo: and we would use those to identify related bugs
(04:14:29 PM) JFo: now, let me get back to subsystem tagging :)
(04:14:54 PM) JFo: https://wiki.ubuntu.com/Kernel/Tagging is the exhaustive listing of tags currently used by the team
(04:15:23 PM) JFo: there are tags listed from Bugs/Tags/ that have been used for some time
(04:15:38 PM) JFo: and you will also see that there are subsystem specific tags
(04:15:58 PM) JFo: this is an effort for us to break down kernel bugs in to a few subsystem categories
(04:16:16 PM) JFo: this will allow us to triage specific sections that triagers have knowledge in
(04:16:40 PM) JFo: while also enhancing our documentation of those subsystems
(04:17:38 PM) JFo: this will have the added benefit of helping those individuals interested in a particular subsystem to focus their energy and move their knowledge forward as they learn more about the kernel
(04:18:10 PM) JFo: i hope for this to empower community members to learn more about the bugs they file too
(04:18:54 PM) JFo: Please keep in mind that we are currently refining our wiki documentation, so these links I am giving you are probably going to change in the near future
(04:18:59 PM) JFo: the content on them, that is
(04:19:22 PM) JFo: any questions on the subsystem tags?
(04:19:46 PM) JFo: ok, we will move on to statuses in that case.
(04:20:15 PM) JFo: most of you are familiar with the bug status, "NEW", "INCOMPLETE", etc.
(04:20:44 PM) JFo: what you may not be aware of, is how I use those to process kernel bugs programmatically
(04:21:42 PM) JFo: given that at any particular time we currently have over 6000 bugs open against the kernel, it is not realy possible for me to provide a focused answer to each.
(04:21:56 PM) JFo: this is where the arsenal scripts i use come in handy
(04:22:10 PM) JFo: and also, in some cases, cause me headaches ;-)
(04:22:54 PM) ext2 is now known as ext3
(04:22:54 PM) JFo: the scripts are meant as a method to keep information current in bugs as to testing based on updates we pull in from upstream stable and Debian
(04:23:11 PM) JFo: as well as determining what information is missing from a bug and requesting that
(04:24:06 PM) JFo: some people get annoyed with me for sending them an automated response, and I'd prefer not to have to, but this is just not possible.
(04:24:46 PM) JFo: i am working on a definitive page identifying the benefits of using the scripts, but i have not yet made it available
(04:25:05 PM) JFo: I hope to do so and link to it from the automated response as soon as next week.
(04:25:36 PM) JFo: this page should also provide some further information on the scripts themselves and what they do
(04:25:54 PM) JFo: any questions on the statuses?
(04:27:28 PM) ClassBot: charlie-tca asked: should we as triagers be using triaged, or leave that one for you to set?
(04:27:48 PM) JFo: charlie-tca, I am perfectly happy for you to set bugs to triaged
(04:28:12 PM) JFo: the only thing i want for them is that they have all the logs, have checked upstream kernels and latest development
(04:28:13 PM) JFo: :)
(04:29:11 PM) JFo: my goal is for most of the bugs to be in a triaged state so that they can be pinged when there are relevant updates to the kernel that may affect the specific issues encountered.
(04:29:31 PM) JFo: ok, Kernel triage summit
(04:29:53 PM) JFo: This will be like an ubuntu user day session in the -classroom channel here
(04:30:31 PM) JFo: but it will be geared toward the subsystems, and providing those of you interested with a deeper understanding of what information is helpful for what subsystems
(04:31:00 PM) JFo: my goal is to provide a bit of professional development for those of you interested in a specific piece of the kernel
(04:31:24 PM) JFo: we will have sessions much like these on bugs in graphics, sound etc.
(04:31:55 PM) JFo: and there will be representatives from the kernel team, and hopefully some upstreams so that there will be greater coverage for Q&A
(04:32:14 PM) JFo: one of the main focuses of this summit will be knowledge and documentation
(04:32:44 PM) JFo: I want the interested parties to broaden their understanding, while at the same time fleshing out or subsystem wiki pages.
(04:33:08 PM) JFo: This will most likely be held at the end of August  or the first part of September
(04:33:13 PM) JFo: any questions on that? :)
(04:33:57 PM) JFo: ok, I'm going to add a point that I missed earlier
(04:34:05 PM) JFo: the Live ISO testing images
(04:34:15 PM) ext2 is now known as ext3
(04:34:41 PM) JFo: It has been my goal for a while now to provide a testing ISO based off of the daily LiveISO image
(04:35:03 PM) JFo: this ISO will boot straight into the desktop as the ubuntu user
(04:35:20 PM) JFo: and will provide a series of tests against the most common kernel issues we see
(04:35:46 PM) JFo: it will enable LoCo teams to do localized testing of the machines available at meetings and Global days
(04:36:06 PM) JFo: while at the same time providing the ubuntu kernel team with much needed information on failures
(04:36:32 PM) JFo: my goal, thanks in very large part to the efforts of bjf
(04:36:56 PM) JFo: is to have these building daily by the end of next week so that we can test them out and start making them available
(04:37:07 PM) JFo: whether or not that happens... we shall see :-)
(04:37:17 PM) JFo: any questions on the Triage Summit?
(04:37:29 PM) JFo: or the Live ISO testing images
(04:38:12 PM) JFo: I'll take that to mean I have covered these topics thoroughly :-D
(04:38:42 PM) ClassBot: simar asked: good work for Live ISO testing images. Great Idea. will this be available on internet.
(04:38:56 PM) JFo: they will be available via the kernel PPA
(04:39:21 PM) JFo: I will also have a wiki page describing them and how to create and submit tests for inclusion in the test suite
(04:39:52 PM) JFo: there will also be a description of the current tests that are available
(04:40:22 PM) ClassBot: simar asked: we will be really happy to test !!!
(04:40:27 PM) ClassBot: charlie-tca asked: they will include a daily kernel ?
(04:40:32 PM) JFo: simar, :-)
(04:40:59 PM) JFo: charlie-tca, they will have what is available to the Live ISO, so it will be the most current development kernel
(04:41:30 PM) JFo: along with all other available development packages, with the notable exception of Open Office
(04:41:34 PM) JFo: and a few others
(04:41:57 PM) JFo: this is mainly to allow us to add the testing stuff and not go over our space allowance
(04:42:27 PM) JFo: a brief idea of the tests we have currently are: suspend/resume
(04:42:34 PM) JFo: video modesetting
(04:42:39 PM) JFo: sound
(04:42:44 PM) JFo: recording
(04:42:48 PM) JFo: among others
(04:43:02 PM) JFo: all things that are built on the kernel or have kernelspace items
(04:43:28 PM) JFo: any other things you want to talk about?
(04:44:10 PM) JFo: ok, well, thanks for listening and asking questions. It is a lot of information to parse :)
(04:44:20 PM) ClassBot: penguin42 asked: Which X version is on those disks? Latest for the distro or an edgers?
(04:44:45 PM) JFo: penguin42, it will be whatever is rolled for the daily ISO
(04:44:52 PM) JFo: I couldn't say for sure
(04:45:12 PM) JFo: I'll hang around until the end of the session if you have further questions.
(04:45:31 PM) JFo: after this, i am always available in the #ubuntu-kernel channel along with the rest of the team
(04:45:42 PM) JFo: thanks everyone :-)
(04:49:58 PM) JFo: I should also mention, the Triage Summit idea came from sconklin :)
(04:55:11 PM) JFo: Thanks folks! I really appreciate all of your questions! :-)
(04:55:18 PM) JFo: see you next time.