Dev Week -- Kernel Triage -- JeremyFoshee -- Wed, Jul 14th, 2010
(04:00:58 PM) JFo: Hi folks, I'm JFo (Jeremy Foshee) and I am the Kernel Bug Triager (04:01:18 PM) JFo: Today we are going to discuss a bit about triaging kernel package bugs. (04:01:45 PM) JFo: this class is going to be something of a follow along for the chat from Saturday's Ubuntu User days (04:01:58 PM) JFo: logs can be found here for that talk: https://wiki.ubuntu.com/UserDays/07102010/What%20is%20a%20kernel,%20and%20why%20do%20i%20need%20it (04:02:11 PM) JFo: so several of the things I'd like to cover are: (04:02:28 PM) JFo: 1) duplicate bugs and the Ubuntu Kernel (04:02:45 PM) JFo: 2) The subsystem tagging of kernel bugs (04:02:59 PM) JFo: 3) triage statuses and the kernel bugs (04:03:10 PM) JFo: and 4) The Kernel Triage Summit (04:03:29 PM) JFo: please feel free to ask questions as you see fit by prefacing them with QUESTION: (04:03:41 PM) JFo: those of you using Lernid will not need to enter that (04:03:48 PM) JFo: ok, duplicates :) (04:04:27 PM) JFo: For those of you who may not be aware, the Ubuntu kernel team have decided not to continue using duplicates in the current way (04:05:02 PM) JFo: this means that, where you would normally have marked bugs that were on the same laptop model, we'd now ask that you do not (04:05:41 PM) JFo: this is due to several things, most notably, the fact that not all of the same laptop models carry the exact same chipsets (04:06:02 PM) JFo: in some cases bus chips and minor board processors could be from different manufacturers etc. (04:06:21 PM) JFo: we have found that minor differences such as these cause us issues when solving bugs (04:06:38 PM) JFo: as we have many cases where a fix for one user did not solve other affected users (04:07:16 PM) JFo: it was decided that we prefer for anyone affected by a bug to file a ticket for their own issue. (04:07:37 PM) JFo: as such we have asked the launchpad team to provide us a way to turn off the ability to mark duplicates (04:07:57 PM) JFo: and we are working on ways to preclude apport from recommending apparently related tickets (04:08:46 PM) JFo: while this will add a heavy load to myself and the team, we feel it is imperative that we are able to work seemingly related issues differently (04:09:05 PM) JFo: this will also allow us to get a better picture of who is affected by a bug and in what way (04:09:22 PM) JFo: any questions on the change to the policy on dupes? (04:09:43 PM) JFo: ok, moving on to item 2 :-) (04:09:54 PM) JFo: subsystem tagging (04:10:02 PM) ClassBot: penguin42 asked: OK, so what do you do when you think you have 50 related bugs - do you hae some common bug to attach them all to? (04:10:39 PM) JFo: penguin42, no, in that case we would take a look at all of the bugs to identify underlying similarities (04:10:58 PM) JFo: we'd then try to fix the underlying problem versus trying to fix symptoms (04:11:27 PM) JFo: this would allow us to narrow our work to address the issue in a better manner. (04:11:47 PM) JFo: It is also important to note that it is not the primary goal of the team to solve issues upstream (04:12:11 PM) JFo: but this has the added benefit of helping us identify what upstream bugs could be related to a particular group of issues (04:12:21 PM) ClassBot: simar asked: Regarding your duplicate policy, if we have duplicate bugs still the developers can have bugs grouped so they can work on like workable bugs in one go? (04:12:48 PM) JFo: simar, we'd prefer not to use the duplicate feature in that way (04:13:22 PM) JFo: in a perfect world, we could identify which bugs are actually duplicates and mark them such, but that takes a lot of overhead (04:13:42 PM) JFo: in my perfect world, each of our bugs has an upstream bug watch :-) (04:14:11 PM) JFo: and we would use those to identify related bugs (04:14:29 PM) JFo: now, let me get back to subsystem tagging :) (04:14:54 PM) JFo: https://wiki.ubuntu.com/Kernel/Tagging is the exhaustive listing of tags currently used by the team (04:15:23 PM) JFo: there are tags listed from Bugs/Tags/ that have been used for some time (04:15:38 PM) JFo: and you will also see that there are subsystem specific tags (04:15:58 PM) JFo: this is an effort for us to break down kernel bugs in to a few subsystem categories (04:16:16 PM) JFo: this will allow us to triage specific sections that triagers have knowledge in (04:16:40 PM) JFo: while also enhancing our documentation of those subsystems (04:17:38 PM) JFo: this will have the added benefit of helping those individuals interested in a particular subsystem to focus their energy and move their knowledge forward as they learn more about the kernel (04:18:10 PM) JFo: i hope for this to empower community members to learn more about the bugs they file too (04:18:54 PM) JFo: Please keep in mind that we are currently refining our wiki documentation, so these links I am giving you are probably going to change in the near future (04:18:59 PM) JFo: the content on them, that is (04:19:22 PM) JFo: any questions on the subsystem tags? (04:19:46 PM) JFo: ok, we will move on to statuses in that case. (04:20:15 PM) JFo: most of you are familiar with the bug status, "NEW", "INCOMPLETE", etc. (04:20:44 PM) JFo: what you may not be aware of, is how I use those to process kernel bugs programmatically (04:21:42 PM) JFo: given that at any particular time we currently have over 6000 bugs open against the kernel, it is not realy possible for me to provide a focused answer to each. (04:21:56 PM) JFo: this is where the arsenal scripts i use come in handy (04:22:10 PM) JFo: and also, in some cases, cause me headaches ;-) (04:22:54 PM) ext2 is now known as ext3 (04:22:54 PM) JFo: the scripts are meant as a method to keep information current in bugs as to testing based on updates we pull in from upstream stable and Debian (04:23:11 PM) JFo: as well as determining what information is missing from a bug and requesting that (04:24:06 PM) JFo: some people get annoyed with me for sending them an automated response, and I'd prefer not to have to, but this is just not possible. (04:24:46 PM) JFo: i am working on a definitive page identifying the benefits of using the scripts, but i have not yet made it available (04:25:05 PM) JFo: I hope to do so and link to it from the automated response as soon as next week. (04:25:36 PM) JFo: this page should also provide some further information on the scripts themselves and what they do (04:25:54 PM) JFo: any questions on the statuses? (04:27:28 PM) ClassBot: charlie-tca asked: should we as triagers be using triaged, or leave that one for you to set? (04:27:48 PM) JFo: charlie-tca, I am perfectly happy for you to set bugs to triaged (04:28:12 PM) JFo: the only thing i want for them is that they have all the logs, have checked upstream kernels and latest development (04:28:13 PM) JFo: :) (04:29:11 PM) JFo: my goal is for most of the bugs to be in a triaged state so that they can be pinged when there are relevant updates to the kernel that may affect the specific issues encountered. (04:29:31 PM) JFo: ok, Kernel triage summit (04:29:53 PM) JFo: This will be like an ubuntu user day session in the -classroom channel here (04:30:31 PM) JFo: but it will be geared toward the subsystems, and providing those of you interested with a deeper understanding of what information is helpful for what subsystems (04:31:00 PM) JFo: my goal is to provide a bit of professional development for those of you interested in a specific piece of the kernel (04:31:24 PM) JFo: we will have sessions much like these on bugs in graphics, sound etc. (04:31:55 PM) JFo: and there will be representatives from the kernel team, and hopefully some upstreams so that there will be greater coverage for Q&A (04:32:14 PM) JFo: one of the main focuses of this summit will be knowledge and documentation (04:32:44 PM) JFo: I want the interested parties to broaden their understanding, while at the same time fleshing out or subsystem wiki pages. (04:33:08 PM) JFo: This will most likely be held at the end of August or the first part of September (04:33:13 PM) JFo: any questions on that? :) (04:33:57 PM) JFo: ok, I'm going to add a point that I missed earlier (04:34:05 PM) JFo: the Live ISO testing images (04:34:15 PM) ext2 is now known as ext3 (04:34:41 PM) JFo: It has been my goal for a while now to provide a testing ISO based off of the daily LiveISO image (04:35:03 PM) JFo: this ISO will boot straight into the desktop as the ubuntu user (04:35:20 PM) JFo: and will provide a series of tests against the most common kernel issues we see (04:35:46 PM) JFo: it will enable LoCo teams to do localized testing of the machines available at meetings and Global days (04:36:06 PM) JFo: while at the same time providing the ubuntu kernel team with much needed information on failures (04:36:32 PM) JFo: my goal, thanks in very large part to the efforts of bjf (04:36:56 PM) JFo: is to have these building daily by the end of next week so that we can test them out and start making them available (04:37:07 PM) JFo: whether or not that happens... we shall see :-) (04:37:17 PM) JFo: any questions on the Triage Summit? (04:37:29 PM) JFo: or the Live ISO testing images (04:38:12 PM) JFo: I'll take that to mean I have covered these topics thoroughly :-D (04:38:42 PM) ClassBot: simar asked: good work for Live ISO testing images. Great Idea. will this be available on internet. (04:38:56 PM) JFo: they will be available via the kernel PPA (04:39:21 PM) JFo: I will also have a wiki page describing them and how to create and submit tests for inclusion in the test suite (04:39:52 PM) JFo: there will also be a description of the current tests that are available (04:40:22 PM) ClassBot: simar asked: we will be really happy to test !!! (04:40:27 PM) ClassBot: charlie-tca asked: they will include a daily kernel ? (04:40:32 PM) JFo: simar, :-) (04:40:59 PM) JFo: charlie-tca, they will have what is available to the Live ISO, so it will be the most current development kernel (04:41:30 PM) JFo: along with all other available development packages, with the notable exception of Open Office (04:41:34 PM) JFo: and a few others (04:41:57 PM) JFo: this is mainly to allow us to add the testing stuff and not go over our space allowance (04:42:27 PM) JFo: a brief idea of the tests we have currently are: suspend/resume (04:42:34 PM) JFo: video modesetting (04:42:39 PM) JFo: sound (04:42:44 PM) JFo: recording (04:42:48 PM) JFo: among others (04:43:02 PM) JFo: all things that are built on the kernel or have kernelspace items (04:43:28 PM) JFo: any other things you want to talk about? (04:44:10 PM) JFo: ok, well, thanks for listening and asking questions. It is a lot of information to parse :) (04:44:20 PM) ClassBot: penguin42 asked: Which X version is on those disks? Latest for the distro or an edgers? (04:44:45 PM) JFo: penguin42, it will be whatever is rolled for the daily ISO (04:44:52 PM) JFo: I couldn't say for sure (04:45:12 PM) JFo: I'll hang around until the end of the session if you have further questions. (04:45:31 PM) JFo: after this, i am always available in the #ubuntu-kernel channel along with the rest of the team (04:45:42 PM) JFo: thanks everyone :-) (04:49:58 PM) JFo: I should also mention, the Triage Summit idea came from sconklin :) (04:55:11 PM) JFo: Thanks folks! I really appreciate all of your questions! :-) (04:55:18 PM) JFo: see you next time.