RatingsAndReviews
37688
Comment: "Flag" -> "Report"
|
37423
heading levels
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
''This is a living specification for ratings and reviews in [[SoftwareCenter|Ubuntu Software Center]]. A branch implementing this feature should not be merged to trunk until the server side is implemented and reliably maintained.'' | ''This is a living specification for ratings and reviews in [[SoftwareCenter|Ubuntu Software Center]]. |
Line 24: | Line 24: |
== Submitting a review == | = Submitting a review = |
Line 82: | Line 82: |
=== to Twitter and/or other sites too === ''This feature should replace the “Share…” link on the software item screen.'' |
== to Twitter and/or other sites too == |
Line 113: | Line 111: |
=== in another language === | == in another language == |
Line 121: | Line 119: |
=== with empty review text or summary, or no rating === | == with empty review text or summary, or no rating == |
Line 127: | Line 125: |
=== containing too much text === | == containing too much text == |
Line 144: | Line 142: |
=== containing a very long word === | == containing a very long word == |
Line 148: | Line 146: |
=== containing inappropriate whitespace === | == containing inappropriate whitespace == |
Line 153: | Line 151: |
=== when there is no Internet connection === | == when there is no Internet connection == |
Line 160: | Line 158: |
=== for many programs in quick succession === | == for many programs in quick succession == |
Line 167: | Line 165: |
=== for the same software a second time === | == for the same software a second time == |
Line 174: | Line 172: |
== Correcting a review == | = Correcting a review = |
Line 181: | Line 179: |
== Presenting reviews == | = Presenting reviews = |
Line 222: | Line 220: |
=== from different versions of the software === | == from different versions of the software == |
Line 227: | Line 225: |
=== containing very long words === | == containing very long words == |
Line 232: | Line 230: |
=== containing inappropriate whitespace === | == containing inappropriate whitespace == |
Line 242: | Line 240: |
=== containing other-directional text === | == containing other-directional text == |
Line 247: | Line 245: |
=== in different languages === | == in different languages == |
Line 260: | Line 258: |
=== when there are many reviews === | == when there are many reviews == |
Line 267: | Line 265: |
=== when there is an error downloading them === | == when there is an error downloading them == |
Line 272: | Line 270: |
=== when there is no Internet connection === | == when there is no Internet connection == |
Line 296: | Line 294: |
== Flagging a review as inappropriate == | = Flagging a review as inappropriate = |
Line 319: | Line 317: |
== Moderating reviews == | = Moderating reviews = |
Line 350: | Line 348: |
== Sorting items by rating == | = Sorting items by rating = |
Line 354: | Line 352: |
== Future work == | = Future work = |
This is a living specification for ratings and reviews in Ubuntu Software Center. Ratings and reviews are a way for Ubuntu users to communicate with other Ubuntu users about software. They are not specifically a means of communication with Ubuntu developers or application developers. Rating and reviewing software involves three software components: Ubuntu Software Center itself the review service, an Internet server that receives, stores, and publishes reviews the Ubuntu Single Sign-On service. Use cases not specified yet Use cases currently specified Contents
This process begins when you activate the “Review…” button on an “Installed Software” item screen. Ubuntu Software Center should first ask the Ubuntu Single Sign-On service to ensure you are signed on, using the summary text “To review software, you need an Ubuntu Single Sign-On account.”. Once you are signed in, a dialog should appear for submitting the review. This dialog should not have a parent window, so that you can refer to other reviews and other items while writing your review. This dialog should be resizable; resizing it should resize only the review text field. That field should be focused by default. The dialog’s title should be of the form “Review {Item Title}”. Within the dialog, first should be displayed the icon, title, and version number of the software. Next should be a “Review by:” item showing your name as registered with the Single Sign-On service (as a subtle hint that you are accountable for your words). This label should act as the accessible label for the review text field, though visually it should be far enough away from the field to look like a separate element. The review text field should accept multiple paragraphs, and should use automatic spell-checking. The “Summary:” field should accept a single line of text. The “Rating:” field should consist of five stars plus a caption. By default, no stars should be filled. Whenever the field is focused, the Left and Right keys should increase or decrease the rating in one-star increments, and the 1 through 5 keys should change the rating to that many stars. The caption should reflect the star you are mousing over, if you are, or otherwise the rating you have currently set: “” (no rating yet), “Awful” (1), “Poor” (2), “Adequate” (3), “Good” (4), or “Excellent” (5). Setting a rating level, whether using a pointing device or a key equivalent, should highlight the new rating in a similar visual style as when clicking a checkbox or radio button. Above the commit buttons should be legal small print: “By submitting this review, you agree not to include anything defamatory, infringing, or illegal. Canonical may, at its discretion, publish your name and review in Ubuntu Software Center and elsewhere, and allow the software or content author to publish it too.” In the code, this string should have a comment that it should not be revised without approval from a lawyer. While the review service is in beta, this should be followed by a second paragraph stating: “Thanks for helping us test the review service. While it is in beta, we can’t guarantee that all reviews will be retained.” The “Cancel” button should have Esc as its keyboard equivalent. The “Publish” button should have Enter as its keyboard equivalent. Choosing “Publish” should: Stopping and errors should similarly be handled exactly as for the sign-on dialog. Submitting a review should send this information to the review service: The review service should return whether the review has been accepted, held for approval, or rejected. If the review is accepted or held for approval, the review dialog should close, USC should automatically navigate to display the relevant software item screen (if it was not being shown already), that screen should scroll smoothly to reveal all of your review, and the background color of the review should fade from a “new” color to the standard color.
If the available version of libgwibber is at least 0.1.0, and libgwibber reports that you have any broadcast accounts that can post text ( Until libgwibber provides a way to determine the character limit for every account, USC should assume a 140-character limit for every account. The the string “reviewed ” (which includes a trailing space), the title of the item, the string “: ” (which includes a trailing space), your rating, shown as five U+2605 and/or U+2606 star characters, followed by a space, your review summary, If not all these elements will fit in the character limit, the post should consist of as much as will fit into one For example: reviewed TreeView X: ★★★☆☆ The best phylogenetic utility I’ve seen in 30 years of teaching high-school s… http://apt.ubuntu.com/p/treeviewx If you chose to post the review to one or more broadcast accounts, USC should not attempt to do that until it has posted successfully to the review service (otherwise it would be posting a link to a review that wasn’t there). While USC is posting to the broadcast accounts, the progress text in the review dialog should be “Posting to {Name of Service}…”, and the review dialog should not disappear until the review has been posted successfully to all of them. If posting to the review service succeeds, but posting to one or more of the broadcast accounts fails, an alert should appear with the review dialog as its parent, no title, primary text “There was a problem posting this review to {comma-separated names of services that failed}.”, secondary text explaining the error if possible, and buttons “Cancel” and ”Try Again”. If you choose “Try Again”, the alert should close, and USC should try to post the review to all services which failed last time. If you choose “Cancel”, both the alert and the review dialog should close.
For now, Ubuntu Software Center should assume that you are writing the review in the same language Ubuntu is using in its interface, and it should specify that language when submitting the review. The review service should store the language specified for every review.
The “Publish” button in should be sensitive only when there is text in both the review field and the Summary field and you have set a rating. The review service should reject with an error any review that does not have all of review text, summary, and a rating.
Review text should have at most 5000 characters, and summaries should have at most 80 characters. In Ubuntu Software Center: Test case: Paste 5027 characters into the review text field. The number “-27” should appear in red. Hold down Backspace. The number should steadily increase. After it passes zero, it should steadily get blacker. On the server side, the review service should reject with an error any review that has too many characters in either field.
There should be no limits on the length of the words used in a review. Any extra word breaking needed should be done when displaying reviews.
There should be no limits on the whitespace used in a review. Any whitespace collapsing needed should be done when displaying reviews.
Whenever there is no Internet connection, the “Publish” button in any review dialog should be insensitive, and at the opposite side of the dialog should be a mini error icon and the text “Not connected to the Internet.”. If there is a connection error of any sort while submitting a review, the “Publish” button should become sensitive again (unless there is no Internet connection), and at the opposite side of the dialog should be a mini error icon and the text “Couldn’t connect to the review service.”. That text should disappear when you next use any control in the dialog (for example, change the contents of any of the fields, or click “Publish” again).
Ubuntu Software Center should not attempt to limit the number or speed of your reviews. (Any attempt at flooding will likely be done outside USC anyway.) The review service should handle flooding appropriately — for example, by silently ignoring excessive numbers of reviews from the same person, and/or by automatically flagging them for review and possible mass deletion.
If you try to review an item and Ubuntu Software Center knows that you have already reviewed that version of that item, it should behave as if you are correcting the review.
If you choose to correct one of your reviews, Ubuntu Software Center should pre-fill the review dialog with the review text, summary, and rating from that review. Whenever all of them are exactly the same as they were originally, the “Publish” button should be insensitive. If a revision is submitted, the review service should treat it as a replacement for the previous review. If someone has flagged a review for moderation, editing it should not cause the review service to clear those flags, but it should record the fact that the review was edited after the flagging.
The average rating for a software item, as shown to someone running a particular Ubuntu release, should be the mean of the last 50 ratings for versions of the software that are less than or equal to the latest version available for that Ubuntu release. Mathematicians How can we improve this algorithm? We want to weight ratings for recent versions more heavily than ratings for old versions. We also want to avoid having most averages settle around 3 stars. Throughout the “Get Software” section, a list view item should have at its trailing end a hyperlink consisting of stars for the average rating for the item, plus text of the form “17 reviews”. Activating the hyperlink should navigate to the software item screen and scroll to its “Reviews” section. In addition, text of the form “ ( When an item is being installed or removed, the progress bar in the list view should replace any rating/reviews link. Similarly a software item screen should have in its top trailing corner a hyperlink of exactly the same form, which does the same thing. In the “Installed Software” section, this area should instead have a “Write a Review…” button that begins the review process. The software item screen in both the “Get Software” and “Installed Software” sections should also contain a “Reviews” section. If there are no reviews for the item, it should consist of a single paragraph with the sentence “None yet.” If you have installed the current version of the software now or in the past, this should be followed by the link “Be the first to review it”. Otherwise, it should consist of the five most relevant reviews for the item, followed by a “Write your own review” link. “Most relevant” in this sense means, of all the reviews you have not flagged, any review that you have submitted previously (regardless of version), plus the most recent reviews for the current version, with any remainder being the most recent reviews of the next most recent version, then the next most recent, and so on. In the “Get Software” section, an item that has any reviews in a software list view should have a link consisting of (a) stars representing the average rating and (b) light text of the form “7 reviews”. Activating the link should navigate to the software item screen and scroll to its Reviews section. The caption below the review should depend on whether USC knows if you’ve given feedback on the review. Other differences for a review you wrote, when you are viewing it signed in, should be that: the “Inappropriate?” link is instead “Edit • Delete” links. Choosing “Write your own review” when you have already reviewed this version of the software, or choosing “Edit” on your review, should display the interface for correcting the review. Choosing “Inappropriate?” or “Delete” should display the relevant interface for flagging the review.
If (and only if) the reviews shown contain any reviews of a version other than the current one, the “Reviews” section of the screen should have subheadings for “Reviews of this version (
If any word in a review is longer than a whole line, Ubuntu Software Center should break the word at the end of the line. The main pane should not scroll horizontally.
When Ubuntu Software Center presents a review, any series of consecutive whitespace characters that: contains at least two line breaks of any sort ( Test case: Submit a test review that contains two paragraphs separated by 50 carriage returns and a few tab characters. When displayed, the two paragraphs should be separated only by a blank line.
Ubuntu Software Center should present each review using the direction customary for the language it is in. For example, an Arabic review should be presented from right to left.
Opposite the “Reviews” heading, in the software item screen, should be an unobtrusive option menu. The menu should contain items for any languages and language families that have any reviews for this software item. The order of the items should be: your locale’s language/family, For example, if your locale is pt_BR, and the current item has four reviews, three pt_PT and one en_AU, the menu should have items “Todas as línguas”, “Português”, a separator, and “Inglês”. The initial setting for the menu should be your language or language family. If you change the setting, a spinner should appear next to the menu, until the reviews in the newly selected language/family are loaded and replace those currently displayed. If the replacement reviews fail to load, the menu should return automatically to its previous setting. The menu selection should persist across items and across USC sessions.
When there are more than ten reviews, in your current language selection, for the same version of an item as is available to you, Ubuntu Software Center should display only the most recent ten. Below them should be a button, as wide as the whole reviews column but with subtle background and border, labelled “Show more reviews”. Activating the button should make it insensitive, change its label to “Loading…”, and load and fade in the next batch of reviews, below the ones that are already displayed. If there are even more reviews available, the button should slide down to make room for the new reviews, change its label back to “Show more reviews”, and resume sensitivity. Otherwise, the button should fade away completely before the reviews fade in.
If Ubuntu Software Center tries to download a batch of reviews and fails (because the connection is interrupted, or it times out, or whatever), the label for loading more reviews for that item should change to read “There was a problem downloading reviews. Try again?”
Ubuntu Software Center should update and cache the average rating, and the number of reviews, for all items in a software source whenever it updates the list of software in that source. It should update and cache the average rating, the number of reviews in each language, and the batch of most recent reviews for your current language setting, for an individual item whenever you visit the screen for that item and there is an Internet connection. The cache should have a maximum size. If there is no Internet connection when you navigate to a software item screen for which there are reviews but none of them are cached, the part of the screen that would normally display reviews should instead say simply “Connect to the Internet to see {number of} reviews.”. If an Internet connection starts working while still displaying that screen, the reviews should load and appear in-place without moving the rest of the screen. Similarly, when (but only when) there is no Internet connection while USC is presenting one batch of reviews and the next oldest batch is not cached, the button to load more reviews should be insensitive, and its label should be “Connect to the Internet to see more reviews.”.
The caption below a review should be: if no-one has given feedback on the review, “Was this review helpful? Yes / No” if you are the only person to have given feedback, “You found this review helpful. Undo” or “You found this review unhelpful. Undo” if you and others have given feedback, then of the form “X of Y people found this review helpful, including you. Undo” or “X of Y people found this review helpful; you did not. Undo” if others but not you have given feedback, then of the form “X of Y people found this review helpful. Did you? Yes / No” If you are not signed in, choosing to mark a review as helpful or not (or choosing “Undo”, if you became signed out since it was displayed) should invoke the Ubuntu Single Sign-On authentication/process, using the summary text “To give feedback on a review, you need an Ubuntu Single Sign-On account.”. Once you are signed in (or if you were already signed in), there should be no extra interface; your decision should take effect immediately, and the caption should update in place. If you wrote the review, your feedback should have no effect.
Flagging a review should first invoke the Ubuntu Single Sign-On authentication/registration process, if necessary, using the summary text “To flag a review as inappropriate, you need an Ubuntu Single Sign-On account.”. The dialog should also feature a “Just Hide It” button; activating this button should cancel the process, but Ubuntu Software Center should then also immediately hide the review and remember indefinitely that you do not want to see that review again. Once you are signed in, if the review turns out to be your own one — or if you were already signed in, and chose “Delete” — USC should present a simple confirmation alert, “Are you sure you want to delete your review?”, with “Cancel” and “Delete” buttons. Otherwise, USC should open a parent-less dialog should appear for flagging the review. It should have a “Why is this review inappropriate?” menu with choices “Unspecified”, “Offensive language”, “Infringes copyright”, “Not about this software”, and “Other”. The “Cancel” button should have Esc as its keyboard equivalent. The “Report” button should have Enter as its keyboard equivalent. It should be sensitive only if the reason is not “Unspecified” and you have entered text in the details field. This dialog should handle progress and HTTP errors in the same way as the review dialog. If you choose “Just Hide It” or “Report”, the review in question should fade away indefinitely, and anything below it should slide up into place.
At any time, the review service should be able to operate in either of two modes: active or passive moderation. In active moderation, every review must be approved manually before appearing to other users, but may still be flagged after that. In passive moderation, we rely on flagging to tell us if a review should be removed. Erratum: “Approve” and “Delete“ should be “Keep Review” and “Delete Review”. https://developer.ubuntu.com/reviews should be a page with access restricted to the moderation team. The page should present the first five items in a list consisting of, in order: In the initial version, each flagged item should show: To maximize speed of moderating, if JavaScript is available, the “Keep Review” and “Delete Review” buttons should not reload the page. They should instead remove that flagged item from the list, with any items below it sliding up, and the next item loading and appearing asynchronously at the bottom of the list. Where an item has been flagged by more than one person, the “Flagged by” bar should contain “+
http://evanmiller.org/how-not-to-sort-by-average-rating.html
The moderation system should be enhanced as necessary to deal with the number of reviews: Reviews for items that are no longer available, or that are available only in obsolete Ubuntu versions, should eventually be expunged.
Was this review helpful? Yes/No
Atom feed? JSON? What extra legal requirements are there?
* Expand the existing concept of PersonalStanding to R&R? * Since users will abuse R&R to report bugs, maybe we can expose to the driver a "turn this review into a bug report" (or link, or dup it, etc). * quickly bug# (via jdo, this is the awesomeness) * Architectural ideas * I want to see my reviews immediately but I can't see yours for 3 hours can be solved on the client (again, couchdb fits in nicely) * Pretty safe bet that this will be huge * A user can only review once? Supercede? * Couch farm to aggregate as necessary to handle load, but LP sees one thing (the root of the tree) * Notifications - we need to think about it * We happen to be the magic conduit for users to be talking to other users * Reviewing the reviews - not for Lucid
Data doesn't need to be up to date, but does need to include reviews Show the latest
analogous to YouTube's "ratings have been disabled for this video" "Help" > "Review This Application..." Show popularity Would be much more useful if we knew
• what to allow • where allow changes? • do we need a new LP page or can/should we re-use https://launchpad.net/ubuntu/lucid/+package/update-manager ? • data needs to be per-binary package, per-distro release (per arch?) • how should s-c get the read-only data? ∘ ratings are small -> can go into a index file (small) ∘ reviews can be potentially very big if we allow multiple ones • offline capability? for cdrom only media ?
mvo and barry worked out an implementation design.
Submitting a review
to Twitter and/or other sites too
if it will all fit;
in another language
with empty review text or summary, or no rating
containing too much text
containing a very long word
containing inappropriate whitespace
when there is no Internet connection
for many programs in quick succession
for the same software a second time
Correcting a review
Presenting reviews
from different versions of the software
containing very long words
containing inappropriate whitespace
e.g. U+00A, U+000C, or U+000D) should be treated as a blank line and nothing else. containing other-directional text
in different languages
e.g. “English”; when there are many reviews
when there is an error downloading them
when there is no Internet connection
Giving feedback on a review
Flagging a review as inappropriate
Moderating reviews
Why on developer.ubuntu.com? Because it will be the central place for application developers. They’ll sign in to see the status of their applications, both before and after they are accepted into the archives — including any reviews. Sorting items by rating
Future work
Rating a review itself
Right of reply for developers
Presenting reviews on an external Web site
Old notes
Launchpad registry
What a review contains
Sending the data from Ubuntu Software Center
Getting the data into Ubuntu Software Center
you just did Other issues
as well as ratings? What the Launchpad team needs
What the Ubuntu Software Center engineers need
Further work
Discussion topics
Implementation design
SoftwareCenter/RatingsAndReviews (last edited 2015-04-13 08:09:55 by mpt)