Ubuntu Wiki

Ubuntu Open Week - Bughelper - Making Bug Work Easier - Markus Korn - Fri, May 2, 2008

=== jcastro changed the topic of #ubuntu-classroom to: Ubuntu Open Week | Information and Logs: https://wiki.ubuntu.com/UbuntuOpenWeek | How to ask questions: https://wiki.ubuntu.com/UbuntuOpenWeek/Rules | Ask questions in #ubuntu-classroom-chat, prefaced with "QUESTION:" |See https://wiki.ubuntu.com/UbuntuOpenWeek/JoiningIn to filter out channel noise |

[15:55] <jcastro> thekorn_: 5 minutes!

=== jcastro changed the topic of #ubuntu-classroom to: Ubuntu Open Week | Information and Logs: https://wiki.ubuntu.com/UbuntuOpenWeek | How to ask questions: https://wiki.ubuntu.com/UbuntuOpenWeek/Rules | Ask questions in #ubuntu-classroom-chat, prefaced with "QUESTION:" |See https://wiki.ubuntu.com/UbuntuOpenWeek/JoiningIn to filter out channel noise | "Bughelper - Making Bug Work Easier" - Markus Korn

[15:56] <jcastro> Ok everyone welcome back, we'll get started in a few minutes

[15:56] <jcastro> Please note that there has been a last minute scheduling change: https://wiki.ubuntu.com/UbuntuOpenWeek/

[15:57] <jcastro> We apologize for this being last minute, so to fill in the time we'll continue to discuss bughelper and general Q+A things like 5-a-day.

[15:57] <jcastro> then @ 1700 UTC I will hold a general Ubuntu Q+A session

[15:58] <jcastro> thanks to everyone for showing up, thekorn_ will begin at the top of the hour!

[15:58] <jcastro> as always, feel free to send me feedback at jorge (at) ubuntu.com

[16:01] * thekorn_ rings the bell

[16:02] <thekorn> Good morning everybody to today's first session of the Ubuntu Open Week.

[16:02] <thekorn> My name is Markus Korn, I'm one of the developers of bughelper and python-launchpad-bugs

[16:02] <thekorn> and I would like to show you how this tools can help you work with your bugs.

[16:03] <thekorn> If you have any questions please ask in #ubuntu-classroom-chat

[16:03] <thekorn> If you want to tryout the examples I provide in this session, please install bughelper and python-launchpad-bugs.

[16:04] <thekorn> Some of this examples will only work with the latest version of both tools.

[16:04] <thekorn> To get this version please add

[16:04] <thekorn> deb http://ppa.launchpad.net/bughelper-dev/ubuntu hardy main

[16:04] <thekorn> to your /etc/apt/sources.list, and replace 'hardy' with the version of ubuntu you use.

[16:05] <thekorn> Then you can should update via 'sudo apt-get update' and install bughelper and python-launchpad-bugs.

[16:05] <thekorn> That's all preparation we need, let's start the session!

[16:05] <thekorn>

[16:06] <thekorn> So first of all: what's bughelper and python-launchpad-bugs?

[16:06] <thekorn> * python-launchpad-bugs is a python interface to work with bugs, and blueprints, in Launchpad.

[16:06] <thekorn> * While bughelper is a suite of utilities that uses python-launchpad-bugs to help you find specific bugs in Launchpad.

[16:07] <thekorn> I'll start off talking about the bughelper suite and then move on to python-launchpad-bugs.

[16:07] <thekorn>

[16:07] <thekorn> The most important tools in the bughelper suite are buginfo, bugnumbers and bughelper itself.

[16:08] <thekorn> Let's have a closer look at this tools:

[16:08] <thekorn>

[16:08] <thekorn> 'buginfo' provides an easy interface for accessing information about a single bug for example

[16:09] <thekorn> 'buginfo --title 123456'

[16:09] <thekorn> will return the bugs title.

[16:09] <thekorn>

[16:09] <thekorn> 'bugnumbers' is designed to return a list of bug based on a query.

[16:10] <thekorn> Let's say you would like to get a list of all bug in ubuntu's 'bughelper' package, run bugnumbers like:

[16:10] <thekorn> 'bugnumbers -p bughelper'

[16:11] <thekorn> As there is also a project called 'bughelper' registered on launchpad.net,

[16:11] <thekorn> you can add the option '-U' to get a complete list of all open bugs in the package and in the project.

[16:12] <thekorn> Other possible options are:

[16:12] <thekorn> '-D=<distro>' to change the distrubution to look in

[16:13] <thekorn> (note: this distro has to be registered in launchpad.net)

[16:13] <thekorn> '-P=<project>' to only search for all bugs in a given project

[16:13] <thekorn> '-l=<url>' to filter buglists based on a given url

[16:14] <thekorn>

[16:14] <thekorn> So far you get a list of all open bugreports,

[16:14] <thekorn> by adding additional options you can filter this lists further.

[16:15] <thekorn> Some of these filters are are also available via the search feature of launchpad, some of them are not.

[16:15] <thekorn> Some features not in Launchpad are querying on the quantity of duplicates, subscribers, comments, attachments.

[16:16] <thekorn> Or you can query on the date and author of the last comment in a bugreport.

[16:16] <thekorn>

[16:16] <thekorn> Let me give you some simple examples:

[16:16] <thekorn> 'bugnumbers -P python-launchpad-bugs --reporter thekorn'

[16:17] <thekorn> 'bugnumbers -P python-launchpad-bugs --reporter thekorn --status New'

[16:17] <thekorn> both calls will return you a list of all open bugs in the python-launchpad-bugs project,

[16:17] <thekorn> but the second one will only show you the 'New' ones.

[16:18] <thekorn>

[16:18] <thekorn> That's no magic so far, let's have a look at a more complex one:

[16:18] <thekorn> 'bugnumbers -l "https://bugs.launchpad.net/~thekorn" --na ">1" --closed'

[16:19] <thekorn> This will return a complete list of all bugs related to me with one or more attachments.

[16:19] <thekorn> This list will also contain closed bugreports

[16:20] <thekorn>

[16:20] <thekorn> On http://people.ubuntu.com/~brian/reports/ Brian Murray is using bugnumbers to create interesting reports.

[16:20] <thekorn> For example 'yesterday' contains a list of all bugs in ubuntu which where reported yesterday.

[16:21] <thekorn> This reports show you that there are different output-formats, in this case he is using a html-format for the output,

[16:21] <thekorn> there are also options for a 'wiki' and 'csv' styled output.

[16:21] <thekorn>

[16:22] <thekorn> Any questions so far?

[16:22] <thekorn> <qense> QUESTION: How do you use bughelper on websites? You need to place it in cron?

[16:23] <thekorn> qense, yes, bdmurray is running a cron job to create this pages,

[16:24] <thekorn> but there is also a script called 'bughelper-server.py' in the bzr branch

[16:24] <thekorn> which helps you to organize runs of bughelper and bugnumbers a bit

[16:25] <thekorn> you can get the branch via:

[16:25] <thekorn> bzr branch lp:bughelper

[16:25] <thekorn> qense, does this answer your question

[16:26] <thekorn> ?

[16:26] <thekorn> ok, let's move on

[16:26] <thekorn>

[16:26] <thekorn> With 'bughelper', the application, you can search bug reports, comments and attachments for a specific string.

[16:27] <thekorn> 'bughelper' provides you clue based on the search result.

[16:27] <thekorn> Clues are stored in clue files with the suffix '.info' in '~/.bughelper/packages'

[16:27] <thekorn> https://code.launchpad.net/bughelper-data/ is the project in launchpad to organise this clue-files and every member of the bugsquad can commit clue files to the .main branch of this project.

[16:28] <thekorn>

[16:28] <thekorn> So, what's the structure of such an clue-file?

[16:28] <thekorn> And how does such an file look like?

[16:28] <thekorn> As an example let's take a look at ~/.bughelper/packages/bughelper

[16:29] <thekorn> (you can view this file online at http://tinyurl.com/2zwm7a)

[16:29] <thekorn> Such clue files are in a xml format and can contain multiple clues.

[16:29] <thekorn> Each clue needs at least two elements:

[16:30] <thekorn> '<contains>'

[16:30] <thekorn> this element contains the actual search pattern.

[16:30] <thekorn> It is possible to add logical elements like 'and' or 'or' to such a node.

[16:30] <thekorn> In the first clue you can see an example of using regular expressions as an search pattern.

[16:31] <thekorn> The 'field' attribute specifies which attribute of an bugreport has to be searched.

[16:31] <thekorn> If you run bughelper and one of the clues in an clue files matches the content of the related '<info>' node will be printed as the output.

[16:31] <thekorn>

[16:32] <thekorn> As an example, if you run 'bughelper -p bughelper' you will get an output like:

[16:32] <thekorn> http://launchpad.net/bugs/88102 [[bughelper: Confirmed/Medium],[bughelper (Ubuntu): Confirmed/Undecided]] - tags: xpath or commandline; importance: Undecided

[16:32] <thekorn>

[16:32] <thekorn> With this I would like to finish the first part of this session about the bughelper tools. Any further questions?

[16:33] <thekorn>

[16:33] <thekorn> Ok, then let's have a brief look at the underlieing python modul 'python-launchpad-bugs'

[16:34] <thekorn> afflux> you can test very simple searches by using: bughelper -p package -T package "keyword" "description". This can be very useful for finding duplicates of crasher bugs which differ slightly. I used "bughelper -A -p compiz -T compiz workaroundRemoveFromFullscreenList 'dup of bug 183685'" for finding a lot of duplicates of this bug

[16:34] <thekorn> thanks afflux, good point, forgot to tell you about this.

[16:34] <thekorn>

[16:35] <thekorn> good, than I think it's time for python-launchpad-bugs

[16:35] <thekorn>

[16:35] <thekorn> python-launchpad-bugs allows you to access bugs.launchpad.net via python.

[16:36] <thekorn> This python module is used by many tools like apport, ubuntu-dev-tools and of course bughelper and bugnumbers.

[16:36] <thekorn>

[16:36] <thekorn> Let me give you a short "Howto" on using python-launchpad-bugs.

[16:36] <thekorn> This requires some basic understanding of python.

[16:36] <thekorn>

[16:37] <thekorn> Let's start a python session and do some general preparation:

[16:37] <thekorn> >>> import launchpadbugs.connector as Connector

[16:37] <thekorn> >>> from launchpadbugs.basebuglistfilter import URLBugListFilter

[16:37] <thekorn>

[16:37] <thekorn> so far so good, let's get a list of all open bugs in the bughelper project

[16:38] <thekorn> >>> BugList = Connector.ConnectBugList()

[16:38] <thekorn> >>> bl = BugList("https://bugs.launchpad.net/bughelper/+bugs")

[16:38] <thekorn> Each element of this list has attributes like 'bugnumber', 'url', 'status' and 'importance'.

[16:39] <thekorn>

[16:39] <thekorn> Getting filtered lists is also very easy:

[16:39] <thekorn> >>> bug_filter = URLBugListFilter()

[16:39] <thekorn> >>> bug_filter.add_option("status", ("New","Confirmed"))

[16:39] <thekorn> >>> bug_filter.add_option("importance", ("Medium",))

[16:40] <thekorn> >>> bl = BugList(bug_filter("https://bugs.launchpad.net/bughelper/+bugs"))

[16:40] <thekorn> It's also possible to use any python functions as a filter, but this would go too far right now.

[16:40] <thekorn>

[16:41] <thekorn> Okay, that's all about bug lists for now, for more information please visit http://tinyurl.com/yrmze9

[16:41] <thekorn>

[16:41] <thekorn> Now, let's have a look on how to handle bugreports with python-launchpad-bugs:

[16:41] <thekorn> >>> Bug = Connector.ConnectBug()

[16:41] <thekorn> >>> b = Bug(123456)

[16:42] <thekorn> The argument of Bug() can either be a bug number, an url of a bugreport or an element of a BugList-object

[16:42] <thekorn> There is a huge amount of attributes of a bug-object.

[16:42] <thekorn> You can access almost all information of a bugreport.

[16:42] <thekorn> Let's have a look at some examples:

[16:43] <thekorn> >>> b.url

[16:43] <thekorn> 'https://bugs.launchpad.net/ubuntu/+source/xine-lib/+bug/123456'

[16:43] <thekorn> >>> print b.subscriptions

[16:43] <thekorn> set([<user ken-paulsen (Valyander)>, [...], <user kubuntu-sujee (LinuxLover)>])

[16:43] <thekorn> >>> print b.comments

[16:43] <thekorn> [<Comment #1 by kubuntu-sujee on 2007-07-03 07:17:39 UTC>,[...],<Comment #6 by bersace on 2007-09-30 22:04:49 UTC>]

[16:43] <thekorn> For a list of all attributes of a Bug-object and some examples on using these attributes have a look at http://tinyurl.com/2mboze

[16:43] <thekorn>

[16:44] <thekorn> So far, we have only read bugs, but with python-launchpad-bugs you can also change bugreports in a very easy way!

[16:44] <thekorn> First of all, only registered user can change bugreports in launchpad, so let's authenticate with our account data:

[16:45] <thekorn> >>> Bug.authentication = {"password":"<your-password>","email":"<your-login-email>"}

[16:45] <thekorn> There is also another possibility of authentication: you can use a mozilla cookie-file (this for example works for cookies created by epiphany or firefox < 3.0)

[16:46] <thekorn> >>> Bug.authentication = "/path/to/the/cookiefile/cookie.txt"

[16:46] <thekorn> If you are using Firefox 3.0 'Bug.authentication' also accepts the new .sqlite type of cookie files.

[16:46] <thekorn>

[16:47] <thekorn> Now I want to show you, as an example, how to change the status of a bug:

[16:48] <thekorn> >>> demo = Bug(193948)

[16:48] <thekorn> >>> demo.status

[16:48] <thekorn> 'New'

[16:48] <thekorn> >>> demo.status = "Invalid"

[16:48] <thekorn> >>> demo.commit()

[16:49] <thekorn> Until you call the 'commit()'-method all changes are local, with 'commit()' you try to commit all changes you did locally to launchpad.

[16:49] <thekorn> Like this you can for example change the description, add comments or attachments, subscribe to a bug or change the tags.

[16:49] <thekorn>

[16:50] <thekorn> Any further questions?

[16:50] <thekorn> let's move on

[16:50] <thekorn>

[16:51] <thekorn> Let's have a look at two more complex examples.

[16:51] <thekorn> You can get this scripts here: https://wiki.ubuntu.com/BugHelper/Dev/python-launchpad-bugs/Examples

[16:51] <thekorn> These two scripts show you how to use python-launchpad-bugs to do things you can't do in launchpad itself.

[16:51] <thekorn>

[16:51] <thekorn> Let's say you are a developer of an upstream project in launchpad.

[16:52] <thekorn> When you create a package of your tool and build it for ubuntu, bugs in ubuntu are closed by the (LP: #123456) syntax, but upstream bugs were not.

[16:52] <thekorn> The first example also closes this upstream bugs in your project and adds a comment containing the changelog-entry.

[16:52] <thekorn>

[16:53] <thekorn> I wrote the second example in the last February when we had all these python-central bugreports.

[16:53] <thekorn> This example returns you a list of bugs created in ubuntu after 2008-02-18 and filters this list by the following criteria:

[16:53] <thekorn> * 'pycentral' or 'python-central' in the content of the bugreport

[16:54] <thekorn> * an attachment called 'DpkgTerminalLog.gz' which contains 'pycentral' or 'python-central'

[16:54] <thekorn>

[16:54] <thekorn> As a last example I would like to show you how to create a bugreport with python-launchpad-bugs, it's only one line of code!

[16:55] <thekorn> >>> b = Bug.New(

[16:55] <thekorn> ... product = {"name":"buglog-data"},

[16:55] <thekorn> ... summary = "First part of the session is almost over",

[16:55] <thekorn> ... description = "Ubuntu Open week rocks!")

[16:55] <thekorn> This creates a new bugreport in the 'buglog-data'-project, if you would like to file a bugreport in ubuntu use

[16:55] <thekorn> 'product = {"target": "ubuntu", "name": "my-package"}'

[16:56] <thekorn>

[16:56] <thekorn> Ok, that's all about python-launchpad-bugs for now, time for some questions!

[16:57] <thekorn> <afflux> QUESTION: why is len(launchpadbugs.connector.ConnectBug()(123456).comments) == 0 for me?

[16:57] <thekorn> <afflux> sorry if that's too much support

[16:59] <thekorn> afflux, I think it's a bug or something

[16:59] <thekorn> as the text-mode works

[16:59] <thekorn> will investigate after this session. let's move on with other questions

[17:00] <thekorn>

[17:00] <thekorn> <qense> QUESTION: Do requests with this package require more or less bandwidth than visiting the website? Or is it the same?

[17:00] <thekorn> That's an interesting question,

[17:00] <thekorn> as LP does not provide a scripting database interface yet,

[17:01] <thekorn> python-launchpad-bugs is doing screen scrapping,

[17:01] <thekorn> this means in almost all cases the required bandwidth is the same.

[17:02] <thekorn> I hope that there will be a public database interface to launchpad.net soon

[17:02] <thekorn> this will make a lot of things easier

[17:02] <thekorn>

[17:03] <thekorn> <ScottK> QUESTION: Is there a way via pyhon-launchpadbugs I can get back a list of possible duplicates (like the LP web interface does when reporting a new bug)?

[17:03] <thekorn> ScottK, not yet, but I think this is a very good idea, just made a note on my TODO list

[17:04] <thekorn>

[17:05] <thekorn> Ok, as far as I can see, i answered all questions.

[17:06] <thekorn> Let me end the first part of this session with a few notes:

[17:06] <thekorn> * bughelper has a separate mailing list, bughelper@l.u.c

[17:07] <thekorn> * browse https://launchpad.net/python-launchpad-bugs and https://launchpad.net/bughelper to file bugreports or get the code

[17:07] <thekorn> * for more information check out our wiki pages at https://wiki.ubuntu.com/BugHelper

[17:07] <thekorn> * and last but not least ping me in #ubuntu-bugs

[17:08] <thekorn> please use this if you have any questions,

[17:08] <thekorn> would like to see a certain feature

[17:08] <thekorn> or I you would like to contribute to bughelper and python-launchpad-bugs

[17:09] <thekorn>

[17:09] <thekorn> Ok, I think that's all for the first part of this session.

[17:09] <thekorn> As jcastro mentioned at the beginning,

[17:10] <thekorn> there were some changes to the schedule, some I will be here for another hour

[17:10] <jcastro> We can continue to answer questions.

[17:10] <jcastro> And we can talk about 5-a-day

[17:12] <jcastro> If you haven't heard about 5-a-day: https://wiki.ubuntu.com/5-A-Day

[17:12] <jcastro> The basic idea is for everyone to touch 5 bugs a day

[17:12] <jcastro> During the Reporting Bugs and Bugsquad sessions Brian and Pedro discussed bug filing and triaging teachniques

[17:13] <jcastro> Note that those sessions will be repeated tomorrow @ 1700 and 1800 UTC

[17:13] <jcastro> With 5-a-day we have provided tools to keep track of bugs you touch

[17:13] <jcastro> which get reported to launchpad, and then we generate statistics

[17:13] <jcastro> Current statistics are here: http://daniel.holba.ch/5-a-day-stats/

[17:14] <jcastro> the stats are broken down into individuals and teams

[17:14] <jcastro> so you can register your Ubuntu Local Team and get tracked via stats

[17:14] <jcastro> So for example, the team I belong to, ubuntu-michigan, has helped triage 1015 bugs

[17:15] <jcastro> This is useful because a) It motivates people in my team to get involved

[17:15] <jcastro> b) It provides metrics so we can measure how effective our team is when it comes to contributing to Ubuntu

[17:16] <jcastro> c) It's nice to be able to say "I helped Ubuntu triage 214 bugs" so I sleep better at night.

[17:16] <jcastro> questions so far?

[17:17] <jcastro> < qense> QUESTION: What is the function of a tag?

[17:17] <jcastro> So with the 5-a-day tool you can "tag" bugs with certain tags

[17:17] <jcastro> we use tags to tag bugs that are fixed during a specific event

[17:18] <jcastro> so for example if you participate in a Hug Day, you tag your bugs with the tag for that day

[17:18] <jcastro> this helps us see how effective we are at triaging bugs during Hug Days

[17:19] <jcastro> Note that this is a voluntary project, you don't have to participate, and many people just triage bugs in launchpad without being part of 5-a-day

[17:19] <jcastro> which is fine, but it's nice to get a sample on how we're doing for hugs day

[17:19] <jcastro> Anyone can use the tags in 5-a-day

[17:19] <jcastro> So for example, if you and your friends want to get together and triage bugs

[17:20] <jcastro> We have this: https://wiki.ubuntu.com/RunningBugJam

[17:20] <jcastro> So for example you and friends can have a "Quense's Bug Killing Blastorama" bug jam, tag it in 5-a-day, and it will show up in the statistics

[17:21] <jcastro> As you can see in the lp log: https://code.edge.launchpad.net/~5-a-day/5-a-day-data/main

[17:21] <jcastro> People are committing and sending in data all the time.

[17:22] <jcastro> What I do every day

[17:22] <jcastro> is run the applet in my panel

[17:22] <jcastro> and as I triage/report/comment on bugs, I drag the firefox tab into the applet

[17:22] <jcastro> and the applet reports it up to launchpad.

[17:23] <jcastro> Some days I don't get many done, some days I do more than 5, which is fine also

[17:23] <jcastro> Any more questions?

[17:23] <thekorn> the applet does not only work with firefox but also with all other mozilla based browsers,

[17:24] <jcastro> If you do participate, there are tips on the wiki for generating your own email signature in both plain text and html.

[17:24] <thekorn> and the applet does also work in xfce, and there will be a QT version soon

[17:24] <jcastro> So that people know what bugs you've touched.

[17:25] <jcastro> So in short, if you're going to be triaging and other bug work, we encourage you to sign up for 5-a-day

[17:26] <jcastro> and your friends in your local group

[17:26] <jcastro> kubuntu-de.org is leading the pack with 1292 bugs, but hard work will move your team up the leaderboard!

[17:27] <jcastro> Ok, we have 30 minutes left, any more questions on bughelper or 5-a-day?

[17:31] <thekorn> If there are no questions, I would like to point you to two tools which are using python-launchpad-bugs:

[17:31] <thekorn> First one is 'Gnome Launchpad Bugs Applet'

[17:31] <thekorn> https://launchpad.net/lp-bugs-applet/

[17:32] <thekorn> It's an applet based on Chris Lamb's 'debian-bts-applet'

[17:33] <thekorn> It shows you your pet bugs in a list in your gnome panel

[17:34] <thekorn> you can feed this list with tools like bugnumbers or you can get all bugs of an ubuntu hug-day

[17:34] <thekorn>

[17:35] <thekorn> another one is 'hugday-tools'

[17:35] <thekorn> https://launchpad.net/hugday-tools/

[17:35] <thekorn> this simlifies editing of the hug-day wiki-pages

[17:35] <thekorn> via the commandline

[17:36] <thekorn> so you don't have to edit the wiki pages directly in your browser,

[17:36] <thekorn> the only thing you have to do is:

[17:37] <thekorn> hugday close 123456

[17:38] <thekorn> to mark bug 123456 in lightgreen and add your nickname to this row on the wiki page

[17:39] <thekorn>

[17:39] <thekorn> <qense> should we also talk about the requirements for joining ubuntu-bugcontrol?

[17:40] <thekorn> I suggest moving this to tomorrows sessions with bdmurray and pedro_,

[17:41] <thekorn> as I'm definitely not the right person to talk about this

[17:42] <thekorn>

[17:42] <thekorn> Ok, if there are no questions left,

[17:42] <thekorn> and if there is nothing to add

[17:43] <thekorn> and, of course, jcastro has nothing to add,

[17:43] <thekorn> I think we should all take a small break,

[17:44] <thekorn> and wait for the next session

[17:45] <jcastro> Thanks Markus for showing us bughelper!

[17:50] <thekorn> jcastro, thanks for organizing this week!