Ubuntu Wiki

To be carried out by: Ubuntu QA Team Lead

Goals

• Ensure that all ISOs are suitable for release

• This process should apply to betas, ReleaseCandidates, final releases, and milestones

T minus 14 days

1. Prepare the release test plan
   • Review feature goals targeted for this release which should be covered by validation testing
   • Review critical bugs which should be verified
   • Review installation test cases with Evan Dandrea
   • Review upgrade test cases with Michael Vogt
   • Identify set of locales which are sufficiently well supported to be tested

2. Remind the Canonical distro team to update StaffHardware

3. Issue call for community testers
   • Commitment to be present and responsive during specific times when candidates will be built
   • Requirement for broadband
   • Reminder to download ISOs in advance, so that rsyncing the candidate is fast
   • List of hardware which needs better representation
   • When volunteering, need to provide:
     • Email address
     • IRC nick
     • Hardware platforms
     • Languages

T minus 7 days

1. Should now have a complete list of test cases and testers

2. Populate and prioritize test tracker with all test cases and assign testers

3. Notify testers via email of their assigned test cases and request confirmation of their ability to carry them out
   • Include instructions for carrying out the tests
     • Installation profile
     • Language
     • Hardware
   • Include instructions for reporting bugs which should be considered for targeting
   • Must respond within 3 days or test cases should be reassigned

T minus 4 days

1. Review confirmations and reassign test cases as necessary

Upon notification from release manager of the availability of a candidate build:

1. Establish which test cases must be covered for this build, based on test coverage of previous builds and impact of changes made since
2. Set current milestone in the tracker and ensure that the candidate builds are listed
3. Notify testers on IRC of the candidate, remind of their assigned test cases and request acknowledgement
4. Reassign any test cases for testers who do not acknowledge
5. Monitor test reports and bug references, notifying the release manager of bugs as they are reported
   • Release manager will identify showstoppers and decide whether to roll a new candidate

Tracking test results

1. Post candidate ISOs and test cases in the tracker
2. Announce start of test cycle

   • ISO testing forum

   • ubuntu-devel

   • tracker mass email feature (not yet implemented)
   • IRC
3. Monitor tracker results
   • Ensure that bugs are filed in Launchpad by testers for genuine problems
4. Proceed with testing until validation criteria are met
   • For minimal validation (alphas): at least one successful installation per ISO is sufficient
   • For full validation (beta, RC, final): all test cases must be successful
5. Discuss test results with the release manager, providing details on test coverage

6. Update https://wiki.ubuntu.com/{ReleaseName}/TechnicalOverview/{Milestone}/Testers (ex: https://wiki.ubuntu.com/QuantalQuetzal/TechnicalOverview/Beta1/Testers) with the list of testers for the release

7. Continue to collect feedback on the build after it is blessed

CategoryProcess