Dev Week -- Writing Apport Package Hooks -- bdmurray -- Tue Sep 1st, 2009

16:03:26 - 16:59:29 UTC (Wikified, formated.)

Hi, my name is Brian Murray and I'm a member of the Ubuntu QA team. I'm here today to talk about how you can get higher quality bug reports about packages that you care about. You can do this by writing an apport hook for your particular package.

First off, what is apport?

Apport is a system which intercepts crashes right when they happen, in development releases of Ubuntu, and gathers useful information about the crash and the operating system environment. Additionally, it is used as a mechanism to file non-crash bug reports about software so that we receive more detailed reports.

Let's look at a sample apport bug report - http://launchpad.net/bugs/416701. The bzr package does not have an apport hook but some useful information is still collected. We have the architecture, the release being used, the package version and the source package name. Additionally, in the Dependencies.txt attachment we have information about the versions of packages upon which bzr depends.

• Are there any questions about apport so far? Okay then, carrying on.

So while all of that can be really useful an apport hook for a package allows us to gather specific information for that package. For example, consider a bug report about usplash. usplash has a dedicated configuration file, located at "/etc/usplash.conf", and this would be something quite helpful in debugging a usplash bug report but not very useful in other package bug reports. Apport looks for package hooks in "/usr/share/apport/package-hooks/" for ones named after the package for which they will be used.

Anatomy of a hook

Looking in "/usr/share/apport/package-hooks/" lets take a look at the usplash hook - with the filename usplash.py. The package hooks are written in Python.

We can see that the usplash.py hook imports the apport.hookutils module - "from apport.hookutils import *". hookutils is a collection of readymade and safe functions for many commonly used things. There are functions for attaching a file's contents, getting a command's output, grabbing hardware information and much more.

This package hook is using 'attach_file_if_exists' and 'attach_hardware'. 'attach_file_if_exists' is pretty self explanatory but what does attach_hardware include? Let's look at the hookutils module to find out.

You can use "pydoc apport.hookutils" or you can view it using codebrowse - http://bazaar.launchpad.net/~apport-hackers/apport/trunk/annotate/head%3A/apport/hookutils.py. The 'attach_hardware' function starts at line 72.

As we can see it adds a wide variety of hardware information to a bug report which can be quite useful for some packages like the kernel and usplash! Having the apport package hooks has reduced the amount of bug ping pong necessary to get information out of a bug reporter and having these convenience functions reduces the amount of work it takes to write a package hook.

In addition to 'attach_hardware' and 'attach_file_if_exists' other functions include: 'attach_conffiles', 'attach_dmesg', 'attach_alsa', 'command_output', 'recent_syslog', 'pci_devices' and 'attach_related_packages'. In the event that you have a group of packages that would benefit from a shared convenience function, please file a bug about apport and include the function you'd like added.

• Are there any questions so far?

• <tormod> QUESTION: is there a way to present a question to the bug reporter? tormod: yes, there actually is and this is a new feature of apport I'll be getting to shorty. Shortly even

Back to the usplash hook, we can see that the usplash configuartion file is added like so "attach_file_if_exists(report, '/etc/usplash.conf', 'UsplashConf')". This means that a bug reported about usplash using apport should have an attachment named 'UsplashConf' and it will contain the reporter's usplash.conf file. Looking at http://launchpad.net/bugs/393238 we can see this actually isn't the case. Because usplash.conf is only 4 or 5 lines it ends up getting put into the bug description. However, most items end up getting added as attachments in Launchpad. Regardless the information is still there and can help triaging the bug.

Now that we've covered what a hook can include; lets's look at how to control when hooks are run.

Running a hook

In the totem hook, "source_totem.py", we can see there is a check to see if the hook was called due to a crash - "if report['ProblemType'] == 'Crash':". If it is due to a crash the hook is not run. If someone were reporting a bug, using ubuntu-bug or Help -> Report a Problem, the ProblemType would be Bug.

In this case the totem hook is only ran when the "ProblemType" is "Bug".

• <thekorn> QUESTION: let's say I'm writing an apport hook which adds a log file, should this hook automatically purge sensitive data, what's best practice in such cases? thekorn: let me find an example - I saw a hook that does some scrubbing recently thekorn: there is one in mysql-dfsg-5.1 http://pastebin.ubuntu.com/263396/ you can see it is checking for the password in the configuration file and replaces it before attaching it to the report I think this, scrubbing the data before uploading, is the best idea Now to tormod's question ...

The totem hook is particularly interesting as it is utilizes interactive questions - which is a new feature in apport. You can see how this works by executing "ubuntu-bug totem", but please don't actually report the bug! The totem hook asks questions and runs tests to determine if the bug report is related to alsa, pulseaudio or codecs in gstreamer. Well add of course totem itself. The totem apport hook also sets the affected package appropriately.

The interactive questions can greatly reduce the amount of triaging work required and helps make the initial bug report much more complete. More detailed information regarding how to use the hook user interface can be found in the python help for the available functions: "python -c 'import apport.ui.HookUI; help(apport.ui.HookUI)'".

• Are there any questions about the interactive questions or the convenience functions?

Now that we know a lot of what hooks can do, lets talk about how to write and test one.

After the last Ubuntu Developer Summit I compiled a list of packages that had recently received a fair number of bug reports and subsequently might benefit from an apport package hook. That list can be found at https://wiki.ubuntu.com/QATeam/Specs/IncreaseApportCoverage.

Writing your own

Let's take rhythmbox from that list and write a simple apport hook for it. I say simple because I'm not entirely certain what would be useful but thought these things might be.

• < andresmujica> QUESTION: What's the purpose of the general-hooks available at /usr/share/apport/general-hooks ? andresmujica: these are always run. For example if you look at the automatix.py one you can see ifi certain criteria are met the bug reports are not filed. And the ubuntu.py checks to see if the package is an ubuntu one among other things.

The easiest way of writing and testing a package hook is to put one in "/usr/share/apport/package-hooks" named after the appropriate package. I'll create one called "source_rhythmbox.py".

We'll import hookutils from apport and os, then in the add_info function we'll add the rhythmdb.xml file, if it exists, and gconf settings for rhythmbox. The hook as written looks like http://pastebin.ubuntu.com/262850/. Its really only 2 or so lines of code but now we'll have some potentially useful information in every rhythmbox crash or bug report. After I've put this file in "/usr/share/apport/package-hooks" I can test it using "ubuntu-bug".

After running "ubuntu-bug rhythmbox", apport presents the user with a dialog asking them if they want to send the problem report. In this dialog box we can see the complete contents of the report and if our collected information was attached. I see a key named rhythmdb.xml which contains the same as the one on my system and a key named GconfRhythbox which contains my gconf settings for rhythmbox. Contains the same information that is

If you look at the compiz hook, source_compiz.py, you can see it includes a debugging portion so you can just execute it. However, I personally find this harder to parse. That's really all there is to writing and testing an apport hook!

Packaging

• < andresmujica> QUESTION: Why source_rhythmbox.py and not just rhythmbox.py ? andresmujica: this way any binary package produced by rhythmbox would have the hook run for it. This is less of an issue with rhythmbox but makes a lot of sense with other packages. Like evolution for example. thekorn: the evolution hook is also a good example of data scrubbing.

In the event that you write a hook for a package that you can not upload or need help getting sponsored, please report a bug about the package missing a hook. Then add the hook as a patch (or a merge proposal) and subscribe me, brian-murray, and the appropriate sponsor's team to the bug report. I'll work to ensure that it gets uploaded.

If you need any further help the apport code is very well documented and there are a few wiki pages about it - https://wiki.ubuntu.com/Apport and https://wiki.ubuntu.com/Apport/DeveloperHowTo. Additionally, you can find me in the ubuntu-bugs channel.

• < sbarthelemy> QUESTION: in which package should the new apport-hook go, in apport or in rhythmbox?

Apport hooks are distributed with the package that will use them, er, the package that will benefit from them. That's why I didn't have the mysql hook readily availabe as the package is not installed on my system. Additionally, this make it easier for us as developers as we can just stick them in our packages and not have to muck with apport.

• < openweek4> QUESTION: will apport hooks work with 3rd party applications?

openweek4: I'm actually really not certain about that question. If you look at the ubuntuone-client hook - bugs get reported to Launchpad but about the ubuntuone project. So it might very well be possible. However, apport would have to be able to communicate with the appropriate bug tracking system.

• < sbarthelemy> QUESTION: the apport hook is expected to be included in debian's own rhythmbox too?

sbarthelemy: most of the apport hooks are carried as a diff from what I have seen

Are there any more questions?

Alright well thanks for coming everyone and I hope this was informative. I look forward to seeing some more apport hooks!