Ubuntu Wiki

Ubuntu Open Week - Giving Useful Feedback - Murat Güneş - Wednesday, November 4, 2009

• Time: 21:00 UTC

"Constructive criticism" has come to be cited as the cornerstone of free software development discourse, but the quality and net value of the huge volume of feedback aggregated on mailing lists, bug trackers and forums often leave something to be desired; and this is an area we can improve upon in Ubuntu specifically. In this session I will be sharing tips on making your feedback genuinely useful to various contributors to free software, including developers, bug triagers and designers.

(04:00:48 PM) jcastro: mgunes, introduce yourself and take it away!
(04:01:07 PM) mgunes: jcastro, thanks. I'll take half a minute to tile my windows..
(04:01:30 PM) mgunes: Hi. I'm Murat Güneş - I'm part of the Ubuntu QA team.
(04:01:48 PM) mgunes: In this session, I'll cover good practices in providing feedback of any sort to different participants in free software (such as developers, bug triagers, designers), and Ubuntu specifically, in different contexts.
(04:02:23 PM) mgunes: For the sake of brevity, I'll assume these contexts to be bug reports, development discourse, and design discourse. But as you'll note, the basic principles tend to overlap to a great extent, and apply to other areas as well.
(04:02:53 PM) mgunes: And as should be obvious, this is a much bigger subject than can be covered in an hour-long talk, so I'll try to provide links to some useful documents that those interested in more details can pursue.
(04:03:10 PM) mgunes: Please post your questions, if any, to #ubuntu-classroom-chat at any time you like, and I'll pause a few times to try to answer them.
(04:03:45 PM) mgunes: I'll avoid the classic dictionary definition and Open Week opening: "What is feedback, anyway?" :)
(04:04:02 PM) mgunes: Since, I'll be assuming we're all on the same page here: we're talking about feedback in social systems, and free software specifically - the act of making the output of your experience about a present social product known, in an effort to improve it in the future.
(04:04:41 PM) mgunes: The first context I want to talk about is perhaps the most obvious one: bug reports.
(04:05:30 PM) mgunes: The process of finding, reporting, triaging and fixing bugs in public space is a unique strength of the free software ecosystem.
(04:05:55 PM) mgunes: It's the ace in our hand, which the proprietary, in-house software industry lacks. How fast free software matures mainly depends on how fast it can afford to fail and be fixed, in public and in a distributed way.
(04:06:36 PM) mgunes: This, obviously, is where the value of good bug reports comes in. The more readily actionable bug reports are, and the more efficient the exchange that takes place in them, the better.
(04:07:32 PM) mgunes: I'll share some "absolute minimum" tips on making your bug reports more likely to be attended and acted upon by bug triagers and developers in a timely manner.
(04:08:03 PM) mgunes: 1) Be concise!
(04:08:20 PM) mgunes: Telling the long story of how your video card worked wonderfully with one Ubuntu release, but failed miserably on the next one, and how your friend who has the same card had no problems with either, may be tempting, but resist that temptation when reporting a bug.
(04:09:00 PM) mgunes: If you skip right to the heart of the problem, and just describe that, and add the necessary technical information, that's just enough.
(04:09:32 PM) mgunes: Keep in mind that people who evaluate bug reports have to read through hundreds of lines of bug descriptions every day.
(04:10:15 PM) mgunes: Having to read through long paragraphs to figure out what exactly the problem is can be exhausting, and can cause people to skip your report in favor of others which are brief and to the point, since there are always more than enough bug reports to attend...
(04:11:06 PM) mgunes: If you feel that you *must* tell a long story to put the bug in context, keep the main bug description to a brief summary, and tell the rest in the comments. This way, the main description is clearer and easier to read.
(04:11:40 PM) mgunes: 2) Follow up with your bug report.
(04:12:06 PM) mgunes: Launchpad will subscribe you to new bug reports you file, and send you mail each time someone comments on them.
(04:12:23 PM) mgunes: That is, unless you opt out by unsubscribing, which is rarely a good idea.
(04:13:12 PM) mgunes: Trying different combinations of settings and hardware configurations, trying to reproduce the bug under different conditions, testing proposed patches, responding to questions asked by developers and bug triagers are activities that are as important as the initial act of reporting the bug.
(04:14:08 PM) mgunes: 3) Make sure your bug report contains the required technical information, such as logs and backtraces.
(04:14:39 PM) mgunes: If you use the "Help > Report a Problem" menu item in most default Ubuntu applications to report your bug, or the "ubuntu-bug" command (which are the preferred ways of reporting bugs), the Apport bug reporting tool will take care of this for you.
(04:15:58 PM) mgunes: (For a detailed description of why it's rarely a good idea to report bugs directly at Launchpad, take a look at http://mdzlog.alcor.net/2009/03/31/please-dont-report-ubuntu-bugs-directly-to-launchpad/ .)
(04:16:20 PM) mgunes: Still, there may be cases where you may not be able to use Apport, or where Apport may not collect the required information.
(04:17:15 PM) mgunes: For these cases, there's one URL that everyone serious about helping Ubuntu through filing good bug reports should keep in mind (no excuses!):
(04:17:23 PM) mgunes: * drum roll *
(04:17:29 PM) mgunes: * curtains open *
(04:17:34 PM) mgunes: https://wiki.ubuntu.com/DebuggingProcedures
(04:17:49 PM) mgunes: This beautiful wiki page has links to various documents about debugging different parts of your system, which are actively maintained by developers, QA engineers and dedicated testers.
(04:18:31 PM) mgunes: Whenever you're dealing with a mysterious crash, non-working hardware or problematic software, and you're stuck and don't know where to go, remember this page.
(04:18:58 PM) mgunes: One more thing regarding the use of Apport:
(04:19:08 PM) mgunes: As I said, the best practice is to report bugs using "Help > Report a Problem" or "ubuntu-bug".
(04:19:25 PM) mgunes: But if you've not been able to use these tools, or neglected to do so, but want debugging information automatically attached to your bug report later on, you can use the "apport-collect" command, given that the package you're dealing with has Apport hooks (here's a list of packages that do: https://wiki.ubuntu.com/Apport#Apport%20Hooks%20available).
(04:20:20 PM) mgunes: Just issue "apport-collect #123456", where #123456 is obviously the number of your bug report, and Apport will take care of the rest.
(04:20:34 PM) mgunes: <dscassel> Question: You find a bug, fire up apport and find someone's reported it, with a good debugging info, but nobody's touched it in months.  Beyond clicking "this bug affects me" and maybe leaving a comment, is there anything I can do?
(04:20:50 PM) mgunes: Good question.
(04:21:12 PM) mgunes: It really depends on the nature of the bug, but usually, yes.
(04:22:02 PM) mgunes: One thing you can check is whether the log or backtrace is actually identical to yours.
(04:22:38 PM) mgunes: If not, and the software has changed over the course of the time that the bug has remained idle, you can attach your own debugging information.
(04:23:17 PM) mgunes: In general, whenever you have new, non-redundant debugging information, it's a good idea to attach it to the bug report, since you'll be adding new information that may further help evaluate the nature of the bug.
(04:24:03 PM) mgunes: Another thing you can do is stop by at #ubuntu-bugs and ask for assistance on whether there's something you can do about that particular bug.
(04:24:14 PM) mgunes: <Jesi> Question: generally people don't want bugs to happen, but say you want to look for any possible bugs, is there anyway to do system diagnostics and tsting other than the "System Testing" feature that comes with Ubuntu?
(04:25:47 PM) mgunes: There are various diagnostics tools not associated with Ubuntu, which you may pursue, but the best practical way of finding bugs, barring automated testing, is to actually use and test the software in question.
(04:26:26 PM) mgunes: Since time is limited, I'll conclude this part by citing some must-read literature on the nature of the bug cycle and good bug reporting practices:
(04:26:55 PM) mgunes: "Bug Reporting in Ubuntu" by Bryce Harrington: http://www2.bryceharrington.org:8080/drupal/node/35
(04:27:16 PM) mgunes: "Equilibrium in Free Software Testing" by Matt Zimmermann: http://mdzlog.alcor.net/2009/04/10/equilibrium-in-free-software-testing/
(04:27:42 PM) mgunes: "The Bug Reporting Culture: 10 Things To Avoid, 10 Things You Must Do" by Fabian Rodriguez: http://www.fabianrodriguez.com/blog/2008/01/18/the-bug-reporting-culture-10-things-to-avoid-10-things-you-must-do/
(04:28:46 PM) mgunes: Any other questions?
(04:29:00 PM) mgunes: <Jesi> Question: What if you find a bug, that isn't really associated to any program as far as you know and your not given the option to send a bug report,  like a little display quirk, and you generally end up in the forums, like for example: when I get a notification in GNOME, the display box is further down on the screen than it should be.
(04:30:39 PM) mgunes: It's a good idea to ask in the development and testing discussion forum whether the particular behavior you're getting may be a bug (don't forget to do a search!).
(04:30:48 PM) mgunes: That saves us lots of false positives and duplicates.
(04:31:17 PM) mgunes: If you can't be sure of the right package to report a bug in, take a look at https://wiki.ubuntu.com/Bugs/FindRightPackage .
(04:31:41 PM) mgunes: If that doesn't help, feel free to ask in the forums, the #ubuntu-bugs channel, or the ubuntu-devel-discuss mailing list.
(04:32:09 PM) mgunes: The second context I'll touch on is development discourse.
(04:32:52 PM) mgunes: By this, I mean the general discussion surrounding development decision-making.
(04:34:06 PM) mgunes: Since time is flowing fast, I'll get straight to sharing my tips
(04:34:56 PM) mgunes: 1) Make sure you understand the original rationale.
(04:35:16 PM) mgunes: People often jump to conclusions about technical decisions that they don't agree with. While it's inevitable that there will be lots of technical decisions you don't agree with in a project as large and complex as Ubuntu, keep in mind that every design decision will make sense to someone, at some level, and a lot of consideration and planning goes into each one.
(04:36:08 PM) mgunes: It's very hard, if not impossible, to arrive at a good criticism of any decision without knowing its exact rationale. You may find yourself thinking that you knew what the rationale of a certain unpopular decision was, since lots of people in the "blogosphere", forums, Slashdot comments and the like keep reiterating it, often not very accurately.
(04:36:41 PM) mgunes: But it's always a good idea to be skeptical, and assume that you don't know the exact rationale unless you heard it first hand, from the source. The internet is very good at amplifying and spreading misinformation.
(04:37:21 PM) mgunes: 2) Be results-oriented.
(04:37:32 PM) mgunes: Discussing a critical piece of UI functionality, which compression algorithm to default to, or whether to manage windows this way or that way makes more sense can be very absorbing and even fun in itself.
(04:37:52 PM) mgunes: But it can lead people to losing sight of the tasks and goals at hand as well.
(04:38:18 PM) mgunes: Remind yourself that discussions are not an end to themselves, but a means to an end. And that beyond a certain threshold, every minute spent discussing is a minute spend not working.
(04:38:59 PM) mgunes: 3) Do not assume bad faith.
(04:39:16 PM) mgunes: Unfortunately, it's becoming increasingly common in many development-related discussions that one "side" assumes the other to have negative intentions, or a hidden agenda.
(04:40:32 PM) mgunes: http://en.wikipedia.org/wiki/Assume_bad_faith lists a number of things that, if you find yourself thinking, you should take some time off discussing. It applies pretty well to free software projects as well.
(04:41:19 PM) mgunes: (Warning: some "strong language" there.)
(04:41:30 PM) mgunes: 4) Make an effort to be original.
(04:41:39 PM) mgunes: Knowing about previous criticism and feedback, and trying to avoid rehashing is always a good idea, since people have limited bandwidth in which to deal with feedback, and it's best reserved to original feedback.
(04:42:11 PM) mgunes: Mailing lists are notorious, especially among newcomers, for being hard to search through, but tools such as Gmane (http://www.gmane.org/) and MarkMail (http://markmail.org/) make them easier to search for previous discussions.
(04:42:46 PM) mgunes: It's always a good idea to do a search to find out whether the particular idea you want to put forward has been brought up before.
(04:43:47 PM) mgunes: If there are no other questions, I'd like to continue with the last context I'll cover: design and "artwork" discourse.
(04:44:52 PM) mgunes: Discussions around design often tend to revolve around the notions of "like" and "dislike".
(04:46:05 PM) mgunes: The cases where stating your like or dislike of a particular piece of design is enough, or good feedback, are very rare.
(04:46:32 PM) mgunes: It's always a good idea to accompany statements of like and dislike with reasoning. Why do you not like it?
(04:47:15 PM) mgunes: Stating specifics helps: which parts? What does the design not allow you to do, and what goals in your daily usage does it make harder to achieve?
(04:48:05 PM) mgunes: (It's a good idea to mind the separation between tasks and goals. Goals are things you want to achieve by using the software. Tasks are particular bits you have to perform to reach your goals.)
(04:48:55 PM) mgunes: Many design discussions are concluded with the statement that "Design is subjective", thus there's no wrong or right.
(04:49:52 PM) mgunes: While there's room for subjectivity in design, it's not an arbitrary act of organization and beautification.
(04:50:35 PM) mgunes: It's done with specific goals and requirements in mind, and if those are not satisfied, the design can be criticized for being "wrong".
(04:51:00 PM) mgunes: Hence the tip I want to share: Mind what the design is trying to achieve, the criteria for achieving it, and the rationale for those criteria.
(04:51:28 PM) mgunes: If any of there are lacking in the discussion, or concealed, ambiguous or not well-stated, that's not a good discussion.
(04:51:45 PM) mgunes: Concentrate on obtaining this data first.
(04:53:00 PM) mgunes: Máirín Duffy of Fedora has a recent blog post where she cites some useful resources for evaluating design based on established principles and vocabulary: http://mairin.wordpress.com/2009/10/24/f12-wallpaper-sprinting/
(04:53:44 PM) mgunes: <Jesi> Question: many people are often critical of prototype, alpha and beta releases, even though they are not final and are released to work out the bugs or to brainstorm, how can one get that message across so things can be productive instead of destructive?
(04:55:18 PM) mgunes: People who tend to over-criticize prototypes or early pre-release software tend not to be familiar with how free software development and desing works.
(04:56:52 PM) mgunes: While stressing the fact that "it's alpha" or "it's a prototype" can be useful per discussion, the better long term solution is to "teach people to fish" - to introduce them to the reasons why people release early and often, why prototypes and wireframes are opened to public scrutiny in the first place, etc.
(04:58:48 PM) mgunes: We're almost out of time; I can perhaps answer one more question, if you have one.
(04:59:18 PM) mgunes: <BlackNinjaVirus> QUESTION: Should we always fresh install ? ? ?
(04:59:20 PM) mgunes: No :)
(04:59:32 PM) mgunes: I think that concludes my session. Thanks everyone.
(04:59:40 PM) mgunes: I think that concludes my session. Thanks everyone.