Proposal for a new QATeam Structure V2.0

Introduction

About this document

This document is not to be considered a "counterproposal" but instead an attempt at contributing to the original proposal for a new QA structure. It presents an alternative assessment of the current Quality Assurance scenario in the Ubuntu ecosystem, its weak and strong points, as well as a detailed specification for safe improvement. The word safe was purposely used, as it is an important part of this proposal. All the content is based in the many talks between the U+1 Team founder (Alvaro Leal, aka. Effenberg0x0), Ubuntu QA Community Coordinator, (Nicholas Skaggs, aka. Guitara), and other contributors mentioned at the Acknowledgements section of this document.

While this is certainly a longer and more omprehensive document than most in the FOSS community, the reasos for this are:

• Professionalism: We must avoid, as much as possible, to communicate any level of uncertainty or lack of professionalism to members and the larger community.
• Transparency: The community must be able to evaluate and understand all aspects involved in the strategies specified in this proposal.
• Governance: As a required step towards healthy and efficient governance, proper and thorough documentation of changes and strategies must be adopted.

A short summary is presented next. It is targeted at eventual readers that may only want to get the "big picture" of this specification. Communities, teams, groups members and their management, who may feel a need for more detailed information and/or engage activelly in a discussion for improvement of this proposal, will hopefully find all needed information in the remaining topics.

Contact Alvaro Leal for any further information, inquiries, to suggest improvements and/or provide any feedback.

Summary

Main reason for changes in QA structure

There's a real need to improve QA in Ubuntu.

• The current "Official" QA structure (1) has problems recruiting and retaining members, motivating them and promoting an energetic, sustainable and evolving environment.
• Meanwhile, "unofficial" "QA-Centric" communities (2) have succeeded doing so but, not being under QA management, their full contributing potential to Ubuntu is not explored.

Note: (1) The Official QA Structure will be referred as OQA (short for "Official QA") in this document
(2) "Unofficial" "QA-Centric" communities will be referred as UQA (short for "unofficial QA")

Goals to be achieved through change

Original Proposal: Suggested goals

The original proposal defined a set of 3 goals we must target in a new structure:

• Ease of communication
• Ability to recruit and retain community members
• Ability to scale with growth potential

Proposal V2.0: Only one primary goal

This proposal asseses that no matter what goals and benefits we define as targets for a change in Ubuntu QA, there is only one primary goal at sight:

• Increase the performance and efficiency of Ubuntu QA

According to this view, everything else are strategies, tactics and methods for achieving such goal.

Proposed changes

Original Proposal: Absorb existing UQAs

Basically, the original proposal targets absorbing existing UQAs as a way to:

• Immediately increase the number of OQA members, thus increasing the number of accomplished QA tasks
• Simplify the new QA structure by deprecating UQAs teams

Proposal V2.0: Embrace UQAs and empower them

• Acknowledge UQAs skills in recruiting, retaining and motivating members
• Preserve UQAs structure and processes
• Empower UQAs to do what they currently do even better
• Modify the current OQA stucture, giving UQAs the power to participate in QA management

Comparison of proposal versions

Conclusions

• UQAs are naturally formed communities, a valuable asset for any product, service, brand. They must be preserved
• Risks are high when attepting to touch UQAs strcture and processes. We can achieve even better results by not doing so
• A deep change in OQA structure is needed, in order to properly acommodate and embrace UQAs goals by

The remaining of this documents details these topics, evaluating the potential benefits and risks of both approaches. It also delineates:

• A suggested set of governanve processes for a new OQA structure
• A suggested set of governance processes that may be of use for UQAs that lack formal management (guidelines)

= Detailed Proposal =

The current QA landscape

Original Proposal: Assessment

• The current Quality Assurance landscape for Ubuntu is very distributed and confusing for not only newcomers, but also for those familiar with the project. This lack of central organization and structure has allowed for many teams to be created with overlapping responsibilities but often working in isolation of each other. The resulting landscape is confusing and difficult to understand or influence. This wiki page represents a proposal to consolidate and better define the teams, roles and responsibilities of the QA ecosystem within the Ubuntu community.

Proposal V2.0: Assessment

"QA-Centric" Communities

A multitude of "QA-Centric" teams and communities emerged in the Ubuntu universe. This is not due to failure of Ubuntu "official" and centered QA structure, but a natural and expected occurence in the Open Source environment. Some of the main characteristics of these communities are:

Examples (Some QA-Centric Communities)

This list was compiled by Nicholas Skaggs and originally posted at his Blog.

• A
• B
• C
• D

Media / Environment

• They are typically media-centered, meaning they are attached to the environment they have adopted for communicating and operating in the digital landscape.
• The chosen environment has a critical role in their effectiveness to recruit and retain members. Their members typically prefer one chosen environment over others.
• One downside of attaining to a main or single digital environment is the lack of communication channels to other communities. Many QA-Centric communities have poor or no knowledge at all of other similar groups and of the the QA-Team activities. There may be small comprehension of the "big picture".

=== Activities, Responsibilities and Delegation (roles) ===

• Communities are deeply influenced by their chosen main media/environment. Groups that have chosen very open and interactive ones, such as web forums and IRC channels typically invest a large portion of their resources (members time, content creation efforts, etc) into providing support to ordinary-users and each other.
• Given the lack of knowledge regarding Ubuntu real QA needs and current shortcomings, few engage into activities demanded by the official QA structure.
• The lack of information also creates the inability to esblish priorities, therefore, few communities have defined roles and delegation of tasks.
• Most of these groups have their members operating basically as bug reporters.
• As, naturally, the number of people able to develop software is smaller than that of ordinary users, few communities engage into developing QA-Related tools.

Management, Hierarchy and Processes

• Mostly all groups have formal and informal leadership roles granted to older members. Some of them award it's most active or gifted members with some distinguished position in the hierarchy.
• Formally outlined rules and processes are likely more commonly found at older communities, which gradually developed such processes driven by past conflicts and dispute situations.

Members Engagement

• Mostly all communities experience great commitment by members and an active and energized community environment.
• This is driven by the passion of its members for the chosen media/environment, for the group itself, for the ideals propagated within the community.
• Another important factor driving high engagement rates is recognition within the community or that of (external) ordinary users and other communities (in the case of groups that operate in highly interative environments).

Official QA Structure (QATeam)

Media / Environment

• Mailing-List: Low interactivity and visibility to ordinary-users and potential new members. Promoted low level of integration between current members.

• Web: Set of pages in the Ubuntu Wiki and Blog. Also offer low interactvity and fails to publicise QA-Team no potential new embers of imcrease integration between current ones.

• IRC: Mostly used for meetings, but even so with low pevel of participation. Most members "idle" at it.

=== Activities, Responsibilities and Delegation (roles) === It's activities, according to QA-Team Activities Page, are:

• ISO Testing
• Daily Smoke Testing
• Stable Release Update (SRU) Testing
• Feature Testing
• General Testing
• Application Testing
• Automated Testing
• Bug reporting and management

As described at the QA-Team Wiki, its responsibilities are:

• Assess and communicate Ubuntu's QA needs.
• Work on creating consistent and efficient QA-related policies.
• Help build communities around QA work and run them smoothly.
• Lead Ubuntu's testing activities.
• Provide leadership to Ubuntu QA-related projects.

There are twelve active roles, presented at the QA-Team Key Positions Page. The descriptions of such positions, the work involved in each, the reasons why they are occupied by the current members, their achievements, current status, To-Do list and etc are simply unknown to ordinary users and QA-Contributors.

Management, Hierarchy and Processes

• The QA-Team Launchpad Page lists individuals directly involved in QA-Management. There are 28 approved members and 57 proposed members as of March., 20th, 2012.
• Ubuntu users that contribute to QA have no distinction until (and if ever) approved to such group. Being recognized as a member of this group requires "established record in contributing to QA via QA-Related teams" and being accepted by current members.
• There are 4 administrators in the QA-Team. At least 3 of them are Canonical employees. Other registered members also are Canonical employees.
• There's no record of these individuals activities, achievement, merits, term lenght of management positions, etc.
• No process to allow for participation of QA Contributors in the QA-Team decisions is outlined publicly, therefore, it's nonexistant.There's no transparency as decision making processes are not publicly detailed.
• Online documentation fails to communicate long-term goals, short-term needs, news and status and to leverage new contribution as well as current and new users engagement.

Members Engagement

• Aparently much lower than perceived in other QA-Centric communities.
• Decreasing number of active members.
• Low rates for new members recruitment and retainment.

Summary: Comparison Table

Analysis

Summary of perceptions

Chalenges and limitations of artificialy formed groups

• The QA-Team is an artifically created organization, not a community. It has no identity. In other terms, no group of users is attached to it.
• It is media/environment agnostic, thus exploring less potentially powerful channels to recruit new members.
• There are no public processes, which makes it hard for new or current members to understand it and evaluate how they could fit at all.
• There's no recognition of current contributors work. The only possibility (obtaining a team membership) is currenly unaccessible for most (stablishing records) and the process for doing so is unclear.

Strenghts of Naturally Formed Communities

Fragility of Naturally Formed Communities

Relevance of Media and Online Environment in Community Formation

Absorbing Communities vs Empowering Communities

Real and Viable Short-Term Goals

=== Real and Viable Long-Term Goals === Future Challenges: Engagement and Creativity are Prerequisites

Proposal for Change

== Goals and Potential Pros, Cons and Risks of a QA Reformulation ===

Goals

Pros

Cons

Risks

New Structure

New Scope

QA-Council

QA-Testing-Council

Testing-Group

Testing-Teams

==== QA-Infrastructure-Council

Infrastructure-Group

Infrastructure-Teams

Responsibilities

Governance

Founder / Ownership

Leadership

Current (PP) Leadership

Councils

Composition of the Councils

Current Councils (PP)

Decision Making Processes

Intra-Teams

Intra-Groups

Delegation

Dispute Resolution

Internal Communication

External Communication

Sub-Teams and Members

Sub-Teams and Members Integration

Team Recruitment vs. Member Recruitment

Team Retainment vs. Member Retainment

Membership

Recognition Memberships

Ubuntu Membership via QA

Technical Resources

Launchpad

Mailing-List

Forum

IRC Channel

Wiki

Team Reports

Media

Regularity

International / Localized Content and Team Branches

Risks

• The QATeam that that the current QA landscapeestablished a set of secondary goals and benefits, which will be presentelater in this document.

It is my perception that while it's easy to think of improvements we can define as goals for a new QA structure, the most critical part of any plan in this sense relies in the fact that we is goals we can define t achieve

The

• Reduce overlapping responsabilities and activities, improving efficiency
• Prevent duplicity of work, so that more tasks are accomplished

Assessment of the current QA scenario