Process

Ubuntu Friendly Process

This page explains the process of having a system listed in the Ubuntu Friendly list. One of the main concepts is that "Ubuntu Friendly" won't be a status that a system gains or loses, but that any tested system will appear in the list, with a rating on how well it works with a particular release of Ubuntu.

Core and extra components

The basic idea from where we will define the way a system is tested and have an Ubuntu Friendly rating is the division between core components and extra components.

The details are on the components subpage.

Testing

Ubuntu Friendly tests will be grouped by the component that they test. In order for a submission to be consider valid and accepted in our rating system, all the tests that cover the core components should have been tested (either pass or fail, but tested).

If a submission does not contain results for ALL tests that cover, the system will reject that submission and it won’t count for Ubuntu Friendly status.

Once that the core components have been covered, the submission will be accepted.

Ratings

Rating per submission

The rating for a particular accepted submission (at least all tests for core components) will be determined by the number of components that passed or failed. The rating will be between 0 and 5.

If any of the tests that cover the core components failed, the submission will get a value of 0. Even if any of the extra components worked.

If the all the tests that cover the core components passed, the submission gets a rating of 3.

The two remaining points to get to a rating of 5 are covered by the extra components with a simple rule of three. If a system does not have any extra components, and, again, all tests covering core components passed, the submission will receive a rating of 5.

Examples:

We have a list of 10 core components (UFC) and 7 extra components (UFE) that we care about. Any other component we will list it as additional component (AC).

System

Components

Pass Results

Rating

Laptop1

6UFC, 3UFE, 0AC

5/6 UFC, 3/3 UFE

0

Laptop2

10UFC, 4UFE, 3AC

10/10 UFC, 1/4 UFE

3.5

Desktop1

7UFC, 5UFE, 1AC

7/7 UFC, 5/5 UFE

5

Netbook1

4UFC, 1UFE, 0AC

4/4 UFC, 0/1 UFE

3

Global rating per system

The global rating for a particular system will be the average of all the submission for that particular system.

A user suggestion: would it be possible to have a different aggregation than “average” on the score?
I think either the median (most common result) or a small barplot of results over score
would be much more expressive.

Ubuntu Friendly Website

Each release of Ubuntu will get its own list. By default, the list shown will be for the latest release of Ubuntu.

Systems will be ordered by global rating first, and number of submission after.

Systems with very bad rating will also appear in the list. There will be no boarder line to call a system Friendly or not, it will be just a rating systems

Example:

****************************************************
Ubuntu 11.04 Friendly

Latitude 2120             4.3   (30 submissions)
Vostro                    4.3   (13 submissions)
Acer EeePC (Intel GPU)    3.3   (40 submissions)
Acer EeePC (ATI GPU)      3.2   (28 submissions)
Mac Book Pro              2.9   (3  submissions)
WinPro Laptop             0.3   (50 submissions)     
****************************************************

The representation of the ratings will be graphical (stars, bars, etc). Clicking on any of the results will give us the details of each of the submissions.

Models with different core components will appear in the list as different models. We have not yet decided which are the core components, but stuff like the GPU will be there for sure.

People will be able to filter by model, type of system, minimum rating, etc.

Giving Feedback on Results

Any user will be able to give feedback and comments for a particular system without needing to run the tests. The feedback will be things like “bluetooth is supposed to be working, but it is not working for me”. These comments will be showed on the details of a particular system, but they won’t affect the actual rating of the system.

Ubuntu Friendly Control process

The Ubuntu Friendly Control can use the feedback information (or any other) to change the rating of a particular system.

UFC can manually:

  • mark submissions as non valid (the algorithm won't take those into account)
  • drop the rating of a system

When a suspicious system or submission is noticed by a member of the UFC, he or she will send an email to the UFC mailing list explaining why the submissions should be marked as invalid or the rating of a system dropped.

The discussion will follow up in the mailing list, and the submission or system will be added to the agenda of the next UFC team meeting.

During the UFC meetings, one of the agenda topics will be the review of these suspicious submissions and ratings and they will be voted one by one. As the discussion already happened in the mailing list, not much discussion should be happening during the meeting.

From the people attending the meeting, at least 75% of the people should agree in dropping the rating or marking the submission as invalid, in order to make that happen.

UbuntuFriendly/Process (last edited 2011-08-11 08:20:01 by apulido)