Canonical Kernel Team Bug Day

The focus of today's bug day will be bugs reporting a kernel oops or kernel panic. Each developer has a list of 20 bugs to focus on. The goal for the day is to determine if the bug remains with the latest Karmic Alpha Release and if enough debug information has been gathered.

There is also a community section for anyone else wanting to help out. The community section is composed of bugs that were marked Incomplete but have since received a response. These will have a goal of trying to determine if the bug can be moved out of the Incomplete state or if additional information is still needed. Feel free to drop into the #ubuntu-kernel IRC channel on FreeNode if you have any questions.

Developer Sections: Kernel Oops/Panic

1. Verify the bug has been confirmed against the latest Karmic 9.10 Alpha release:
   1. If not, ask them to test Karmic and if possible also test the latest mainline kernel build. (see Stock Reply)

      • Set status to Incomplete

2. If the bug still remains with the latest Karmic 9.10 Alpha release:
   1. If the entire Kernel Oops or Panic output has been captured and attached to the bug:

      • Set status to Triaged and also set the Importance

   2. If the Kernel Oops/Panic output has not been provided, ask them to do so (see Stock Reply)

      • Set status to Incomplete

Community Section: Incomplete Bugs

1. If the requested information has been provided:
   1. Verify the bug has been confirmed against the latest Karmic 9.10 Alpha release:
      1. If not, ask them to test Karmic and if possible also test the latest mainline kernel build. (see Stock Reply)

         • Status remains Incomplete

      2. If so, set status to Triaged, set the bug's Importance.

2. If the requested info has not been provided:
   1. If the bug has been in an Incomplete state for more than 6 weeks:

      1. Close the bug as Won't Fix and post a comment to confirm the issue exists against the latest karmic 9.10 Alpha release and to also remember to provide the requested information. (see Stock Reply)

3. If the response is that this bug is no longer an issue, close the bug:
   • Examples. . .

     • Set the bug to Fix Released if this was resolved with an update or newer release

     • Set the bug to Invalid if they no longer have the hardware

Tools

Here are references to some useful tools that may help you more efficiently process the list of bugs.

greasemonkey

Launchpad Greasemonkey Scripts modify the look and feel of Launchpad to help one efficiently and effectively triage bugs. The Launchpad Greasemonkey Scripts were recently packaged into a Firefox extension and made available from a PPA:

https://launchpad.net/~gm-dev-launchpad/+archive/ppa

Enable the gm-dev-launchpad PPA (see instructions in link above) and install the firefox-lp-improvements package. For the purpose of Bug Day's, the stockreplies gm-script will be the most useful. The stockreplies script allows you to save custom comments to insert into bug reports and additionally set the status and importance of a bug at the same time.

NOTE: You can point your lp_stockreplies.user.js to http://people.ubuntu.com/~andres.mujica/greasemonkey/kernel-replies.xml where the Kernel standard replies are available.  (Also a kernel-tags.xml is available)

launchpadlib

https://help.launchpad.net/API/launchpadlib

The launchpadlib library depends on wadllib, another open-source library released by the Launchpad team. Get a copy of the launchpadlib and wadllib source with bzr and install it.

•   $ bzr branch lp:wadllib
    $ cd wadllib
    $ sudo ./setup.py install

Then do the same for launchpadlib.

•   $ bzr branch lp:launchpadlib
    $ cd launchpadlib
    $ sudo ./setup.py install

After installation you can write custom scripts to manipulate bug reports. This is useful for dealing with a list of bugs which all require the same type of actions/comment. For example, here is a stock-reply.py script which reads a list of bugs from a "stock.txt" file, posts a standard comment to each bug, transitions the status to Incomplete, and subscribes oneself to the bug report. It's quicker to use a launchpadlib script for this rather than having to do a page load per bug in order to use the greasemonkey stockreplies.

NOTE: In Karmic, python-launchpadlib and python-wadllib are available from main.  In Jaunty you can install them with sudo apt-get install python-launchpadlib  python-wadllib. Some minor changes are available at bzr but for most use cases it would work this way.  If you want to grab the latest version from bzr be aware of Bug 400170 - https://launchpad.net/bugs/400170 

Arsenal Scripts

If you intend to use the Arsenal Scripts (Use them with caution, and take into account that at some point in time some of those would run automatically) follow the Readme file available at http://bazaar.launchpad.net/~arsenal-devel/arsenal/master/annotate/head%3A/README

You can grab them like this:

•   $ bzr branch lp:~arsenal-devel/arsenal/master

Also for the Template-Python dependency be sure to use:

•   $ svn co svn://svn.tt2.org/tt/Template-Python/trunk Template-Python

Stock Replies

Here's some common stock replies that may apply to some of the bugs. Feel free to modify or combine stock replies below to best fit each bug:

Capture Kernel Oops/Panic (Non-frozen system)

NOTE: In Karmic, this should be obsoleted by kerneloops.

It's most helpful to the kernel developers if you can capture the entire contents of the Kernel Oops/Panic and attach it to this bug report.  Since it appears the system is still functioning, it's likely the contents of the kernel oops/panic have been logged to your dmesg output.  To capture your dmesg output, please run the following command from a Terminal (Applications->Accessories->Terminal) after the Kernel Oops/Panic has been triggered.  Then attach the resulting "dmesg.log" file to this bug report.

dmesg > dmesg.log

Capture Kernel Oops/Panic (Frozen system)

It's most helpful to the kernel developers if you can capture the entire contents of the Kernel Oops/Panic and attach it to this bug report.  Unfortunately it's sometimes the case that the system has frozen before you have been able to capture the Oops/Panic.  As a result, the linux-crashdump package has been provided to help capture the contents of a Kernel Oops/Panic, especially in the event of a sudden system freeze/lockup.  You will first need to install the linux-crashdump package, the apport-retrace package, and then reboot the system.

sudo apt-get install apport-retrace linux-crashdump
sudo reboot

Once installed and rebooted, please reproduce the kernel oops/panic (if possible).  Upon the next reboot, run the following command from a Terminal (Applications->Accessories->Terminal):

sudo apport-retrace -s -R /var/crash/linux-image*-.crash > /tmp/crash.log

This collects the contents of the Kernel Oops/Panic and writes it to a file, "crash.log", in your /tmp directory.  Please attach the resulting "crash.log" file to this bug report.  For additional information, please refer to https://wiki.ubuntu.com/KernelTeam/CrashdumpRecipe .

NOTE: If the stack trace isn't captured by runing the apport-retrace command, have the bug reporter run the following instead:

apport-unpack /var/crash/linux-image*.crash /tmp/unpacked crash /usr/lib/debug/boot/vmlinux-uname -r /tmp/unpacked/VmCore

Then run the following commands when at the crash> prompt:

sys > /tmp/crash.log log >> /tmp/crash.log exit

Attach the crash.log file to the bug report.

Test Karmic and Upstream

Can you also confirm this issue exists with the most recent Karmic Koala 9.10 Alpha release?  ISO CD images are available at http://cdimage.ubuntu.com/releases/karmic/ .  If the issue remains with Karmic it would be great to then also test the latest upstream mainline kernel available.  This will allow additional upstream developers to examine this issue.  Refer to https://wiki.ubuntu.com/KernelMainlineBuilds .  Thanks in advance.

• Set status to Incomplete

Close due to lack of info

This bug report was marked as Incomplete a while ago and has not had any feedback to provide the requested information.  As a result this bug is being closed.  Please reopen if this is still an issue in the latest Karmic 9.10 Alpha release http://cdimage.ubuntu.com/releases/karmic/ . Also, please be sure to provide any requested information that may have been missing.  To reopen the bug, click on the current status under the Status column and change the status back to "New".  Thanks.

• Set status to Won't Fix

Bug List

Please refer to the following for the list of bugs to focus on for this bug day:

http://qa.ubuntu.com/reports/ogasawara/kernel-bugday/20090915.html