Process

Differences between revisions 17 and 31 (spanning 14 versions)
Revision 17 as of 2010-06-22 23:29:01
Size: 14290
Editor: c-67-164-40-130
Comment:
Revision 31 as of 2011-02-10 22:23:10
Size: 23
Editor: c-76-112-212-248
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Post Release Application Submission Process =

||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;"><<TableOfContents>>||

'''THIS DOCUMENT IS IN DEVELOPMENT BY jono AT ubuntu DOT com - THIS IS NOT FINAL.'''

The purpose of this document is to outline the process where by applications can be submitted to a community-driven evaluation board for review

== Goal Definition ==

=== Problem ===

The current Ubuntu process for getting an application into an Ubuntu archive is slow and complex and unapproachable for application authors. This complexity of process is preventing application authors and users from making software available in Ubuntu.

=== Solution ===

The high-level solution is to provide a fully transparent facility in which application authors can propose their application for inclusion in the Ubuntu Software Center. This inclusion can happen at any time (including post-release), but only applies to new applications and not to existing software that is present in Ubuntu archives such as main/universe.

== Limitations ==

In the Ubuntu world we already have existing avenues for getting applications into the archive:

 * Core Dev/MOTU developer approval - all core-dev/MOTU developers can upload applications to main and universe respectfully, under the premise it meets the quality needs of Ubuntu.
 * Partner - Canonical provide a partner repository in which ISVs can deliver software.

Each of these repositories is frozen for release and no other software is accepted until the new development release opens. Only updates pertaining to the SRU policy are accepted into stable releases.

The process outlined here is known as the '''Post-Release Application Process''' and has the following attributes:

 * Only new applications that are not present in an existing official Ubuntu repository (such as main/universe) are eligible in this process (e.g a new version of an application in an existing official repository is not eligible).
 * The process only applies to a specific version of an application (e.g. PyJunior 1.0 or PyJunior 1.2).
 * No other software can depend on the application being submitted (e.g. development libraries are not eligible).
 * Only executable applications (and content that is part of them) are eligible, and not content, documentation or media.
 * Applications must be Open Source and available under an OSI approved license.

== Process ==

The process for submitting an application has three primary stages:

 1. Preparation - get your application packaged and available for assessment.
 2. Submission - submit the package to be reviewed for inclusion in the software center.
 3. Assessment - the assessment process in which our Application Review Board decided whether the application is admitted to the software center.

Each stage is broken down in a series of components, as illustrated below:

{{attachment:process.jpg|Process}}

I will now outline each of these different stages.

=== 1. Preparation ===

''This step prepares the application ready for assessment by the Application Review Board''.

This step has three components:

 * '''Package the application''' - package the application ready to be installed and run.
  * The application should be packaged using the standard Debian packaging system. Documentation for this process is in the Packaging Guide (https://wiki.ubuntu.com/PackagingGuide).
 * '''Upload to a PPA''' - the package is uploaded to the user's PPA
  * The package is reviewed from the PPA.
 * '''Testing''' - you should test your application against the Application Review Board's checklist identified below.

By the end of this process the application should be in a form that it can be installed from a PPA, with access to the source package.

=== 2. Submission ===

''The second step is for the developer to formulate a submission to present the application, supplying additional information about the application to the Application Review Board''.

The data to be gathered in this process is:

 * Name (text)
 * Email (text)
 * Package to be assessed (URL to PPA)
 * License (text)
 * Support contact (URL to forum / mailing list etc)
 * Notes (text with information pertinent to the assessment)
 * Optional: Bug tracker (URL)

To file an application, the following process is executed.

 1. User uses the pre-defined wiki template (outlined below) and provides all required content. You should review all content, ensure that you followed the checklist of tests and that everything is in place for a fully completed submission. Incomplete submissions will be rejected.
 1. This is added to http://wiki.ubuntu.com/ApplicationReviews/Apps/<appname-version> (e.g. http://wiki.ubuntu.com/ApplicationReviews/Apps/PyJunior-1.0). A link to this application is placed on http://wiki.ubuntu.com/ApplicationReviews/Queue
 1. A mail is sent (using the email template below) to `app-reviews AT ubuntu DOT com`. This mailing list is where the application is evaluated.
 1. The application review appears in the email moderation queue for the Application Review Board.

=== 3. Assessment ===

''The third step is the assessment process. The Application Review Board (ARB) assesses the application and passes a verdict on it's inclusion''.

For details of the structure of this board, see the 'Application Review Board Codification' section below.

The review process requires an assessment of the application and at least one (1) code review by a member of the ARB. The review process assesses the application against the following checklist:

|| '''Area''' || '''Test''' || '''Pass/Fail''' || '''Comments''' ||
|| Packaging || The application is well packaged using the Debian packaging system || || ||
|| Packaging || All correct dependencies are met || || ||
|| Packaging || Application installs cleanly || || ||
|| Packaging || Application be removed cleanly || || ||
|| Packaging || Includes suitable Copyright and licensing content || || ||
|| Integration || Application integrates into the Applications menu || || ||
|| Run Test || Application runs correctly || || ||
|| Run Test || Major features operate as expected || || ||
|| Content || Includes family-friendly end-user content || || ||

The ARB receives notifications of applications for review via the mailing list and reviews the wiki page that is available from the submission step above. The process in which it is reviewed is:

 1. The ARB review the application based upon the quality criteria outlined above.
 1. Each ARB member votes accordingly with +1 (vote for the approval of the application to be included in Ubuntu), -1 (vote for the application to be rejected), or 0 (no verdict passed).
 1. The results are tallied with the following outcomes:
  * Three or more +1 votes - the application is approved.
  * If there are more than one -1 votes - the application is discussed in more detail and another vote occurs after additional discussion and review has taken place.
  * Three or more -1 votes - the application is rejected.
 1. The result of the vote is added to the application's submission wiki page and the author is emailed with the result.

The entire assessment process is transparent and takes place on a publicly accessible wiki and mailing list.

= Supplementary Documents =

== Application Submission Wiki Template ==

Below is the official template that application authors should use when requesting an application to be reviewed by the ARB:

{{{
= Application Review For <application name> =

It is highly recommended that you subscribe to this page using the 'Subscribe' link above. This will ensure you receive an email when an Application Review Board member updates this page.

Last Updated: DD-MM-YYYY

== About You ==

 * NAME:
 * EMAIL ADDRESS:

The Application:

 * APPLICATION NAME:
 * LICENSE:
 * PPA URL:
 * SUPPORT RESOURCE (URL to forum / mailing list etc)
 * KNOWN ISSUES:
  * Issue.
  * Issue.

=== Application Notes ===

Please add additional notes about this application review below:

= Application Review Board Response =

 * '''MEMBER''': ARB Member's Name
 * '''VERDICT''': (+1 or -1)
 * '''NOTES''': Some additional input on the request.

}}}

== Application Submission Email Template ==

This email template can be used to send the email requesting the submission to app-reviews AT ubuntu DOT com. The subject should be:

{{{
Subject: Application Review Request: <application> <version>
}}}

e.g.

{{{
Subject: Application Review Request: PyJunior 1.0
}}}

Here is the body of the email:

{{{
Hi,

I would like to request an application review of <application> <version>. The submission document can be found at:

  http://wiki.ubuntu.com/ApplicationReviews/Apps/<appname-version>

Thanks!

  <your name>

}}}

== Application Review Board Codification ==

This document aims to:

 * Provide a clear delegation and codification of how applications are assessed for inclusion in the "post release process".

 * Describe clear democratic and meritocratic processes for the appointment of leadership and staff positions in the Application Review Board.

 * Avoid single points of failure.

For active teams and subprojects with Ubuntu, the Ubuntu Community council delegates many of its responsibilities to "Team Councils." These councils act as proxies for the Community Council over a particular team or scope of activity within the Ubuntu community. These governance councils are ultimately responsible for the actions and activity within their team or scope and resolves disputes and manage policies and procedures internal to their team and frequently appoint Ubuntu members on behalf of the CC.

The Application Review Board (ARB) is the team governance council for assessing post-release applications.

=== Application Review Board Charter ===

The ARB is the group that is ultimately responsible for the governing the application review process and how it interfaces with the rest of the Ubuntu community and governance systems. It will:

 * Consist of five to seven members. Membership should be public and published.

 * Decisions will be made by a majority of voting ARB members when at least three and more than half of the total members have voted.

 * ARB members should be accessible by and responsive to the developer community.

 * Hold "meetings" regularly and visibly. Meetings can either be in IRC in the "ubuntu-meeting" channel or in a public mailing list.

 * Be appointed by the Ubuntu Community Council. Nominations would be open and public and would be considered and evaluated by the CC. Each candidate should prepare a wiki page summarizing their nomination and their contributions and including and referencing testimonials (e.g., something similar to what is prepared for Ubuntu membership). The CC commits to evaluating all nominations on the following criteria, listed in order of importance:

  - The nominees active status as an Ubuntu member ''(essential)''.
  
  - The nominees support from ''at least'' one active Ubuntu developer ''(essential)''.

  - Opinions and testimonials (positive and negative) from current Ubuntu members.

  - Clear evidence of activity within the developer community (quality, quantity and duration).

 * Serve terms of two (2) years. ARB members could serve multiple or repeated terms. Weight will be given to proved contributors and reelection of consistently active members should be both easy and common.
 
 * Have a chair with a casting vote, appointed by the Community Council.

The ARB would have a number of rights and responsibilities, and be ultimately responsible for approving quality applications for availability to Ubuntu users. These include:

 * Evaluating applications, including quality assessments, testing, and code review.

 * Resolving disputes in applications as per the existing dispute resolution system.

 * Regularly and when possible (i.e., monthly), sending reports or representatives to CC members to weigh in on issues of membership and to update the council on the ARB business.







= Discussion =

If this idea is successful, I would think there could be a LOT of submissions. Perhaps some sort of filter mechanism (community voting or automatic sorting by some "quality" metric or some such?) could help the ARB in prioritizing if this turns out to happen.

Some thought should be given to security issues, e.g. if someone uploads an app which contains a trojan or something.

The last thing we need is another board IMO, can we do a separate instance of brainstorm and allow Ubuntu members and developers to vote on submissions. It would be a lot more scalable than having a board to handle everything. --fagan

I think it makes sense to have a board to handle conflict resolution, or to override decisions made against community guidelines (basically, as an arbiter of last resort). However, in line with the rest of Ubuntu, we should have a process that is more community based. I would recommend a submission process that allows an X week voting period, with the ability to circumvent that process by appealing to the ARB. We would also allow user reviews of the product, and enough negative rankings (perhaps categorized into quality, security, usefulness, etc rankings) would take an app out of the repository until the problem is addressed. --ilya haykinson

1. I believe you can improve a bit on submission process. A button implemented in PPA (or project page) can bring all the information required for submission, such as names, emails, project/product description, etc. It will get into maillist automatically. And you may get this without subdomain.

2. I think it should be a community process. If ARB would have to decide for each application, then in time due to the sheer number of the submitted applications
   a) ARB would only have time to work on approving/reviewing applications, and nothing else
   b) It will take forever for the application to be reviewed/approved

The community can definitely make this process easier.
-- Alex Lourie
##REDIRECT AppReviews

PostReleaseApps/Process (last edited 2011-02-15 17:50:53 by pool-173-50-130-127)