Summary
The current Quality Assurance landscape for Ubuntu is very distributed and confusing for not only newcomers, but also for those familiar with the project. This lack of central organization and structure has allowed for many teams to be created with overlapping responsibilities but often working in isolation of each other. The resulting landscape is confusing and difficult to understand or influence. This wiki page represents a proposal to consolidate and better define the teams, roles and responsibilities of the QA ecosystem within the Ubuntu community.
Rationale
As we look to bring about better QA practices inside the Ubuntu ecosystem, it's critical we have a team structure that can support and drive these changes. If we wish to communicate clearly and be effective within the greater Ubuntu community, it's imperative we are able to do the same internally.
Goals
This specification wishes to address the following shortcomings within the current enviroment
- Ease of Communication
- Communication both internally between teams and externally to the greater Ubuntu community and world will be much easier inside the new structure. The primary Ubuntu QA team will receive regular communication from the two core sub-teams in order to assess needs and communicate outwards as necessary. In addition, the Ubuntu QA team will be a liaison between the QA community and the rest of the community.
- Ability to recruit and retain community members
- The current on-boarding process for a Ubuntu QA member is both varied and in most cases undocumented and/or without structure. Generally, if someone is interested in QA the best advice to getting involved is to be persistent, skilled, and a little lucky. Instead, we want to build a healthy community that is capable of on-boarding new people and increasing the commitment and responsibility level as they prove themselves. This vetting and sponsorship should be modeled after other similar processes inside Ubuntu, such as becoming a Ubuntu member or a MOTU, etc.
- Ability to scale with growth potential
- On-boarding new members when it happens requires a lot of involvement and attention of individual community members and teams. This new structure will offload much of that work onto a standardized on-boarding process that a potential member can walk through by themselves without diverting community effort until it’s needed in the process. This reduction in effort will stem from having documentation in place making it clear what is required, in addition to having clearer defined teams, roles, and responsibilities.
Use Cases
- Mark is an Ubuntu application developer who has introduced new features for his application and is looking for user feedback and testing. He communicates with the Ubuntu QA team who walk him through the process of adding testcases to test his application and communicate his timeline and needs to the testing sub-team. The testing sub-team executes the testplan, and the Ubuntu QA team is able to share the results and feedback with the developer and the testers themselves.
- Jim is new to the Ubuntu community, but it anxious to help. He subscribes to the ubuntu-qa mailing list and asks how to help start testing. Jim is then shown to the wiki documentation and is mentored through a process whereby he is able to become a trusted member of the testing team. Having developed his skills executing testcases and participating in updating and maintaining testcases, Jim is granted update access to the testcase repository and other infastructure tools.
- Kathy is a curious casual Ubuntu user who attends her loco regularly. She shows up to a global jam session her loco is having and wants to help. Kathy is not familar with IRC or launchpad, nor is she a developer. She does like playing with new things and loads up the latest Ubuntu iso of the current development version. Kathy is greeted with a prompt for her to help test the new version of Ubuntu. She is able to follow along through the tool, hitting next and reporting yes or no failures. The information about Kathy's testing session, the bugs found and her comments are sent back to the testing dashboard for the Ubuntu QA team for further insight.
- Michelle is a reluctant computer user, who likes things to just work and is attractted to the stability and ease of use of Ubuntu LTS. One day her system informs her that her favorite program has crashed and offers for her to report a bug. Michelle is able to click through the prompts and notices a bug report has been created. She now wonders what will happen next to her bug report, and if she is able to do anything to help.
Scope
This specification covers the consolidation of the Ubuntu QA community and many of the teams invovled in QA. However, it is not intended to address the Ubuntu Flavors QA teams. Instead these teams will continue to be run and administered according to the policies and procedures created by the corresponding community and teams.
Design
PLEASE NOTE THIS DOES NOT REPRESENT THE FINAL STRUCTURE AND IS OPEN TO DEBATE. RATHER, THIS IS A STARTING POINT FOR DISCUSSING CONSOLIDATION OF THE CURRENT TEAMS DOING QA WORK INSIDE OF UBUNTU.
Proposed Structure
The proposed structure places emphasis on function, rather than team. This allows for natural cross-project interaction and allows people to match their skill sets to a wider variety of work inside Ubuntu. Listed below are the following roles and examples of the work they would perform. In addition, QA teams who currently perform some of this work are listed.
Roles
System Administrators
- Archive and package maintenance
- Infrastructure support (jenkins, case conductor, iso tracker)
Developers
- Web site development (portals, e.g. Ubuntu Friendly)
- Infrastructure tools development (checkbox, integration scripts to launchpad)
Testers
- New Feature Testing
- Bug Reporting
- Bug Verification
- ISO Testing
- SRU Testing
Technical Writers
- Bug Triage
- Documentation (wiki, best practices/process)
- Creating Test Cases
Advocate
- Marketing/Publicizing/Promoting
- Recruiting/Outreach
Teams
The primary team will continue to be known as Ubuntu QA. Additionally, two sub-teams (infrastructure and testing) will exist with specific focuses, and the potential for further sub-teams within them. As part of this proposal, control/admin teams have been removed and in general all teams should have open membership. Many of the current teams will be migrated under or into one of the infrastructure or testing teams under Ubuntu QA. The complete and final structure will be determined as part of the next steps once the specification is agreed upon. The chart below provides a visual picture of the basic hierarchy.
Following is a concise summary of the core teams and there responsibilities .
Ubuntu QA
- Provide administrative oversight
- Set policy and direction
- Provide single vision for quality in Ubuntu
- Communicate and publicize QA work
- Provide on-boarding process for new members
The Ubuntu QA team will be the primary point of contact for quality assurance inside Ubuntu. Most of the administration required will be done via this team. In addition Ubuntu QA will help communicate direction and help with on-boarding and retaining community members. Ideally a process similar to becoming a MOTU will be followed to help in delegating responsibilities and critical tasks as needed. This process will not be seen as a requirement to participate in QA, merely as a way to obtain greater responsibility and leadership in QA.
Infrastructure
- Develop tools to assist with quality assurance in Ubuntu
- Maintain and operate shared infrastructure tools for Ubuntu QA
- Maintain consistent package archive state, coordinate archive updates
This team will ensure the QA community has the tools it needs to get testing done properly. This includes helping to ensure we have a good testing environment infrastructure, and building tools as needed in support of testing efforts.
Testing
- Perform manual and automated testing on Ubuntu and Ubuntu + 1 releases
- SRU testing
- ISO Testing
- Manual application testing
- New Feature testing
- U+1 testing
- Perform bug reporting
- Perform bug triaging and verification
This team will perform testing of all kinds, with a primary distinction being testing for development versions of Ubuntu and testing for the stable version of Ubuntu. In addition, this team will help shape proper bug reporting processes and help with bug verification and triaging.
Implementation Plan
- Precise Cycle
- Discovery Phase
- Release this proposal and solicit feedback
- Revise and finalize proposal
- Create detailed action plan and timeline
- Create blueprint for work
- Discovery Phase
- Q Cycle
- Implementation Phase
- Execute blueprint
- Communicate and implement basic structure
- Cleanup Phase
- Finalize
- Finalize infrastructure needs
- Deprecate old teams and structure
- Implementation Phase
- R Cycle
- Post Analysis Phase
- Complete transition
- Perform post-migration analysis
- Post Analysis Phase
Implementation
The specifics around how this consolidation will occur should be addressed as part of the detailed action plan. This specificaton represents a proposal and is still subject to change until agreed upon.
Outstanding Issues
Next Steps
Calls for public discussion to occur using the comments on this page, the ubuntu-quality mailing list, and the Ubuntu QA community weekly IRC meeting.