ErrorTracker

Differences between revisions 1 and 2
Revision 1 as of 2011-07-04 17:12:06
Size: 235
Editor: mpt
Comment: + Prior art
Revision 2 as of 2011-07-05 10:31:57
Size: 3276
Editor: mpt
Comment: rationale and cases to sketch
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== Prior art == == Rationale ==

To help Ubuntu reach a standard of quality similar to competing operating systems, developers should spend less time asking for information on individual bug reports, and more time fixing those bugs that affect users most often.

To determine which bugs those are, we should collect crash reports from as many people as possible, before and after release. This means ''not'' requiring them to sign in to any Web site, enter any text, submit hundreds of megabytes of data, receive e-mail, or do anything more complicated than clicking a button. An automated system should then analyze which problems are caused by the same bug. If developers need more information about a particular kind of crash, they should be able to configure the system to automatically retrieve that information when the problem next occurs.

Statistics collected by Microsoft show that a bug reported by their Windows Error Reporting system “is 4.5 to 5.1 times more likely to be fixed than a bug reported directly by a human”, that fixing the right 1 percent of bugs addresses 50 percent of customer issues, and that fixing 20 percent of bugs addresses 80 percent of customer issues.

=== Prior art ===

Windows Error Reporting is perhaps the most advanced crash reporting system. As described in [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.148.716&rep=rep1&type=pdf|K Glerum, K Kinshumann, S Greenberg, et al.: “Debugging in the (very) large: Ten years of implementation and experience”]] (PDF), it uses progressive data collection where developers can request more than the “minidump” if necessary to understand particular problems, and automatically notifies users if a software update fixes their problem. Hardware vendors can [[http://msdn.microsoft.com/en-us/windows/hardware/gg487440|see crash reports specific to their hardware]].

{{attachment:windows-app-progress.gif}} {{attachment:windows-app.gif}} {{attachment:windows-os.png}}

Mac OS X has a Crash``Reporter system that submits crash data to Apple. As described in [[http://developer.apple.com/library/mac/technotes/tn2004/tn2123.html|Technical Note TN2123]], “There is currently no way for third party developers to access the reports submitted via CrashReporter”.
Line 5: Line 19:
{{attachment:windows-app-progress.gif}} {{attachment:windows-app.gif}} {{attachment:windows-os.png}} As a result, some Mac software developers have created their own crash tracking systems, such as [[http://unexpectedlyquit.com/|Adobe’s]] and [[http://www.flickr.com/photos/jfpoole/143205824/|Adium’s]].

Mozilla uses [[https://wiki.mozilla.org/Breakpad/Design|Breakpad]] to collect and submit minidumps on the client side, and [[https://wiki.mozilla.org/Socorro|Socorro]] to analyze and present them on the server side. Anyone can access crash data at [[https://crash-stats.mozilla.com/|crash-stats.mozilla.com]].

== Client design ==

||<tablestyle="width:100%"> ||'''The problem can be reported''' ||'''Your admin has blocked problem reporting'''||
||'''Part of the OS crashes'''||{{attachment:os-crash-reportable.png}}||{{attachment:os-crash-unreportable.png}} ||
||'''An application crashes'''||
||'''An application hangs''' ||
||'''Report is submitted''' ||

Rationale

To help Ubuntu reach a standard of quality similar to competing operating systems, developers should spend less time asking for information on individual bug reports, and more time fixing those bugs that affect users most often.

To determine which bugs those are, we should collect crash reports from as many people as possible, before and after release. This means not requiring them to sign in to any Web site, enter any text, submit hundreds of megabytes of data, receive e-mail, or do anything more complicated than clicking a button. An automated system should then analyze which problems are caused by the same bug. If developers need more information about a particular kind of crash, they should be able to configure the system to automatically retrieve that information when the problem next occurs.

Statistics collected by Microsoft show that a bug reported by their Windows Error Reporting system “is 4.5 to 5.1 times more likely to be fixed than a bug reported directly by a human”, that fixing the right 1 percent of bugs addresses 50 percent of customer issues, and that fixing 20 percent of bugs addresses 80 percent of customer issues.

Prior art

Windows Error Reporting is perhaps the most advanced crash reporting system. As described in K Glerum, K Kinshumann, S Greenberg, et al.: “Debugging in the (very) large: Ten years of implementation and experience” (PDF), it uses progressive data collection where developers can request more than the “minidump” if necessary to understand particular problems, and automatically notifies users if a software update fixes their problem. Hardware vendors can see crash reports specific to their hardware.

windows-os.png

Mac OS X has a CrashReporter system that submits crash data to Apple. As described in Technical Note TN2123, “There is currently no way for third party developers to access the reports submitted via CrashReporter”.

mac-app.png mac-plugin.png mac-os.png mac-hang.png

As a result, some Mac software developers have created their own crash tracking systems, such as Adobe’s and Adium’s.

Mozilla uses Breakpad to collect and submit minidumps on the client side, and Socorro to analyze and present them on the server side. Anyone can access crash data at crash-stats.mozilla.com.

Client design

The problem can be reported

Your admin has blocked problem reporting

Part of the OS crashes

An application crashes

An application hangs

Report is submitted

ErrorTracker (last edited 2018-02-27 11:56:11 by mpt)