MaloneUpstreamUsabilityWorkflow

Malone Usability Report

Status

Introduction

The first adopter of Malone within Canonical was the Bazaar development team. The intent was to release early and often, in the hopes that the feedback from this team of bug tracker power users would provide valuable insight into shaping the direction of Malone development.

This document captures the current state of affairs on the usability of Malone for our upstream users, told in their words.

Rationale

We need to document usability feedback about Malone, using it as a guide for further Malone development.

Method

We observed JamesBlackwell and SteveAlexander using Malone. We shut up and let them do the talking, collecting their thoughts and complaints about the Malone workflow. This is called the think aloud protocol.

Malone Usability Issues

Bug tasks

JamesBlackwell expressed frustration with the bug and task interface -- as someone involved with an upstream product, he doesn't care about distributions. In general, if upstream maintainers and QA people are to use Malone, the time wasted by navigating between bugs and tasks must not just be less than the time saved by recoding fixes that have already been made downstream, it must also seem less. This is a tall order, especially in the short term, when few distributions are using Malone so the probability of mergable fixes (or useful information) appearing in bug reports is very low.

James also had specific comments on the task interface.

  • On a typical viewing of the task page in 800 x 600, all one sees is

the source package name, binary package name, title and description; from the upstream perspective product, maintainer, title and description. "It's important that the bug task page show the most important information nearer the top of the page. For example, information about the priority of the bug should be near the top of the bug task page so that it can be read without needing to scroll. 'Assign milestone' is another thing that should be near the top of the page."

  • The status messages displayed when a change is made currently say "Updated on <some date>", which isn't particularly informative. When I make a change, I want to see a status message showing me what was changed. I often find myself making a change to a task, turning away to do something else, then coming back to the page and forgetting what I'd just changed.

  • Having two IDs (the bug ID and the task ID) displayed on the task page

has also proven to be a constant source of confusion for new and not-so-new users of Malone. James: Bug 449 and task 452 is confusing on the bug task page!

  • "I shouldn't be able to reassign the product."
  • "It's confusing that login is jblack@... and assignee is jblack."
  • "Using a dropdown for assignee that, by default, contains the list of people that have already been assigned to a bug on this thing would be a cool way of indicating that I'm allowed to change the assignee field. Assigning someone outside of that list could be done with a widget beside that dropdown, that might look similar to what is currently used for the assignee field."

These are important problems that need addressing in the very near future, as they echo the sentiments of many new Malone users. The most important of these issues are being addressed in DemystifyingBugsAndBugTasks and will be part of MaloneOneDotZero.

Bug Listing

User feedback tells us that the bug listing page might not be presenting information in a way that shows the user the most important things first, nor that it is presenting each row in a way that works with smaller screen resolutions (i.e. by showing most important row information as the leftmost data in the row.)

JamesBlackwell: I look at the submitter to see how important the bug is (i.e. the submitter field can be a good indication of how much attention I should pay to this bug report.) The bug listing page is too wide; I can't see who's assigned to the bug. I should also be able to see the priority on a bug in 800 x 600 without scrolling. Perhaps critical bugs can be colored red to draw attention to them.

It was also noted that we might want to display more bugs on the bug listing page by default because it's painful and inefficient to have to do a lot of clicking between pages. Editing a bunch of bugs consecutively becomes particularly painful when only 20 bugs are shown per results page.

We saw that the advanced search has some bugs in it, but these are minor issues that can be fixed before MaloneOneDotZero. It was also suggested that it may be useful to use a cookie to save searches without having to login.

The bug listing is being fixed. As for the other problems, while they are important, they will need to be scheduled for after MaloneOneDotZero.

Bug Index Page

The bug index page and the bug edit page appeared to cause our users grief due to various problems in the way it lays things out and by being difficult to make sense of.

JamesBlackwell: The tasks listing is confusing. The page I'm going to end up at depends on where I click on the task row! The "bug tasks" header line on bug page should be highlighted.

SteveAlexander: I clicked under "Assigned to" and expected to end up on a page where I could change the assignee on this task. Instead I ended up on a page (the task page) where the Assignee column can't even be seen without scrolling.

JamesBlackwell: Comments aren't threaded, which makes the discussion hard to follow. I'd like a confirmation after adding a comment (and other things.)

Here's a few other points we noted from James' comments:

  • description shouldn't have a horizontal scroll bar
  • description shouldn't be on both task and bug page
  • it looks odd that description appears once again as a comment
  • don't know what the actions mean (edit, +bugtracker, +CVE, etc.)
  • editing bug details: thought this was the place where one goes to edit what is actually on the task page
  • seeing who's subscribed to a bug is useful, but could become quite long; maybe the list should be collapsed

These are all important issues, but not necessarily required for MaloneOneDotZero. There have been two attempts made so far to improve the bug page, one that attempted to merge the bug and bug task pages and one that simply attempted to shift the information around on the bug page itself. The latter was rolled back because it was a significant change made without having first passed through the approval process. The former was rolled back because that attempt proved to be overwhelming from a usability standpoint.

We need to come up with a solution that doesn't overwhelm the user, and that sabdfl will be happy with.

General Issues

Some last bits that James pointed out:

  • He can't remember the URL to Malone, so has to go to bazaar.canonical.com first. (See also LaunchpadBranding.)

  • Clicks on "malone" in bread crumbs, expects to get to bazaar bugs, where he started
  • Hard to know how to use the system, feels stupid, no help

A comment like the latter indicates that Malone usability is in a state that requires serious attention soon.

Usability Modes

JamesBlackwell proposed the idea of different usability modes: beginner, intermediate, and advanced. By default, Malone could be set in beginner mode, which would provide question mark icons beside various fields and options in the UI, e.g. the bug actions portlet. The beginner mode would offer suggestions about various functions in Malone -- for example, suggesting that if you want to track a bug in an upstream bug tracker, you can add a watch to the bug.

Unfortunately such an approach has several problems. It would require designing and maintaining multiple interfaces for the same tasks, taking more development time. It is not obvious that some functions are "intermediate" or "advanced", so beginners or intermediate users may assume they are not present in Malone at all rather than realizing that they are available in another mode. And people who use intermediate or advanced mode would find it harder to help people using beginner or intermediate mode, because the interfaces would be different.

CategoryUdu CategorySpec

UbuntuDownUnder/BOFs/MaloneUpstreamUsabilityWorkflow (last edited 2008-08-06 16:41:21 by localhost)