Schedule
|
Size: 2360
Comment: moved from Testing/Community
|
Size: 3718
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| == Community ISO testing schedule == | [[Include(Testing/Header)]] |
| Line 3: | Line 3: |
| == i386, Ubuntu erase disk == | ||<tablestyle="float:right; font-size: 0.9em; background:#F1F1ED; margin: 0 0 0 0;" style="padding:0.5em;">[[TableOfContents(2)]]|| |
| Line 5: | Line 5: |
| * '''Primary:''' [https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~alauppe alauppe] * '''Secondary:''' [https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~bryceharrington bryce], [https://launchpad.net/~michl Michl], [https://launchpad.net/~jt-jtholmes jtholmes], [https://launchpad.net/~dg10050 dg10050] |
Testing of the major milestones Beta, RC and Final stretches over about a week and is done in three stages to ensure the best use of available human resources. |
| Line 8: | Line 7: |
| == i386, Ubuntu manual partitioning == | = Hardy ISO test schedule = |
| Line 10: | Line 9: |
| * '''Primary:''' [https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~bryceharrington bryce], [https://launchpad.net/~huygens-25 huygens], [https://launchpad.net/~rubenr-canada rubenr-canada], [https://launchpad.net/~jt-jtholmes jtholmes] * '''Secondary:''' [https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~jwcampbell j1mc] |
||<rowbgcolor="#cccccc"> '''Release''' || '''Task''' || '''Date''' || ||<rowbgcolor="#FFFFCC"> Beta || Smoke testing || March 13-15 || ||<rowbgcolor="#FFFFCC"> || Fix validation testing || March 15-17 || ||<rowbgcolor="#FFFFCC"> || ISO validation testing || March 18-19 || ||<rowbgcolor="#FFEBBB"> Release Candidate || Smoke testing || April 10-12 || ||<rowbgcolor="#FFEBBB"> || Fix validation testing || April 12-14 || ||<rowbgcolor="#FFEBBB"> || ISO validation testing || April 15-16 || ||<rowbgcolor="#FFCCCC"> Final || Smoke testing || April 17-19 || ||<rowbgcolor="#FFCCCC"> || Fix validation testing || April 19-21 || ||<rowbgcolor="#FFCCCC"> || ISO validation testing || April 22-23 || |
| Line 13: | Line 20: |
| == i386, Ubuntu auto-resize == | = Testing details = |
| Line 15: | Line 22: |
| * '''Primary:''' [https://launchpad.net/~michl Michl], [https://launchpad.net/~dg10050 dg10050] * '''Secondary:''' [https://launchpad.net/~mixium mixium], [https://launchpad.net/~alauppe alauppe] |
In the first ''smoke test'' phase we perform a wide series of tests to cover every ISO and install method but do not attempt to test every combination. The aim is to find bugs quickly so they can be addressed for the milestone. As developers fix these bus as well as other milestoned bugs we perform ''fix validation testing'' to confirm that these bugs are indeed fixed and that the fixes have not introduced other problems. Finally we perform complete ISO validation testing in which we apply every relevant test case to each ISO in order to ensure that every ISO we ship works as expected. This phase is quite labor intensive and will require the participation of 20 or more people in a short window of time. |
| Line 18: | Line 24: |
| == i386, Kubuntu erase disk == | == Smoke testing == |
| Line 20: | Line 26: |
| * '''Primary:''' [https://launchpad.net/~firephoto firephoto], [https://launchpad.net/~mib-mibgames mibuk], [https://launchpad.net/~abattoir abattoir] * '''Secondary:''' |
The first round of testing is designed to find high impact bugs in a short period of time so they can be addressed before the milestone is released. Smoke testing should occur just after the archives are frozen. Upgrade and netboot testing can start immediately; ISO testing starts after the first daily images based on the frozen archive appear. The following elements should receive coverage during smoke testing: |
| Line 23: | Line 28: |
| == i386, Kubuntu manual partitioning == | * Core distro flavours: '''Ubuntu, Kubuntu, Edubuntu, Server''' * Supported architectures: '''i386, amd64, sparc''' * Image types: '''Live CD, Alternate, Netboot''' (not DVDs) * Install methods: '''erase disk, auto-resize, manual partitioning, entire disk with encryption, Netboot, Upgrade''' * Select new features: '''Compiz, tracker, Firefox 3, GVFS, X config tool, disk encryption, Active Directory, Wubi''' |
| Line 25: | Line 34: |
| * '''Primary:''' [https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~jwcampbell j1mc], [https://launchpad.net/~huygens-25 huygens] * '''Secondary:''' [https://launchpad.net/~vorian Vorian], [https://launchpad.net/~firephoto firephoto] |
Testing will be coordinated in '''#ubuntu-testing''' and the results tracked on a '''[:Testing/ISO/Smoke:simple checklist]'''. The number of tests is kept low intentionally, but they should be performed with care (e.g. there are only 3 Live CD session test in total, but they should be thorough; testing much of the default applications and installing others). |
| Line 28: | Line 36: |
| == i386, Kubuntu auto-resize == * '''Primary:''' [https://launchpad.net/~vorian Vorian] * '''Secondary:''' [https://launchpad.net/~firephoto firephoto], [https://launchpad.net/~dg10050 dg10050], [https://launchpad.net/~abattoir abattoir] |
== Fix validation testing == |
| Line 33: | Line 38: |
| == WinFOSS == | Test the changes that were introduced after the archive freeze to verify the fixes work and that they don't break surrounding functionality. '''[:Testing/ISO/FixValidation:Track fix validation testing here]''' |
| Line 35: | Line 40: |
| * [https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~michl Michl] | == Image validation testing == |
| Line 37: | Line 42: |
| '''Note:''' ''Please do not add yourself to this page directly, but email iso.testing AT gmail.com (read by Henrik and pochu) so we can coordinate. '' Return to [:/..:Community Testing] |
Complete validation testing of all images and test cases. If phases 1 and 2 have been effective there should not be any show-stopper bugs appearing at this stage. The broader testing is likely to reveal a large number of minor bugs though. These should be targeted at the next milestone or stable release updates. Track image validation testing at '''[http://iso.qa.ubuntu.com/ iso.qa.ubuntu.com]'''. |
Testing of the major milestones Beta, RC and Final stretches over about a week and is done in three stages to ensure the best use of available human resources.
Hardy ISO test schedule
Release |
Task |
Date |
Beta |
Smoke testing |
March 13-15 |
|
Fix validation testing |
March 15-17 |
|
ISO validation testing |
March 18-19 |
Release Candidate |
Smoke testing |
April 10-12 |
|
Fix validation testing |
April 12-14 |
|
ISO validation testing |
April 15-16 |
Final |
Smoke testing |
April 17-19 |
|
Fix validation testing |
April 19-21 |
|
ISO validation testing |
April 22-23 |
Testing details
In the first smoke test phase we perform a wide series of tests to cover every ISO and install method but do not attempt to test every combination. The aim is to find bugs quickly so they can be addressed for the milestone. As developers fix these bus as well as other milestoned bugs we perform fix validation testing to confirm that these bugs are indeed fixed and that the fixes have not introduced other problems. Finally we perform complete ISO validation testing in which we apply every relevant test case to each ISO in order to ensure that every ISO we ship works as expected. This phase is quite labor intensive and will require the participation of 20 or more people in a short window of time.
Smoke testing
The first round of testing is designed to find high impact bugs in a short period of time so they can be addressed before the milestone is released. Smoke testing should occur just after the archives are frozen. Upgrade and netboot testing can start immediately; ISO testing starts after the first daily images based on the frozen archive appear. The following elements should receive coverage during smoke testing:
Core distro flavours: Ubuntu, Kubuntu, Edubuntu, Server
Supported architectures: i386, amd64, sparc
Image types: Live CD, Alternate, Netboot (not DVDs)
Install methods: erase disk, auto-resize, manual partitioning, entire disk with encryption, Netboot, Upgrade
Select new features: Compiz, tracker, Firefox 3, GVFS, X config tool, disk encryption, Active Directory, Wubi
Testing will be coordinated in #ubuntu-testing and the results tracked on a [:Testing/ISO/Smoke:simple checklist]. The number of tests is kept low intentionally, but they should be performed with care (e.g. there are only 3 Live CD session test in total, but they should be thorough; testing much of the default applications and installing others).
Fix validation testing
Test the changes that were introduced after the archive freeze to verify the fixes work and that they don't break surrounding functionality. [:Testing/ISO/FixValidation:Track fix validation testing here]
Image validation testing
Complete validation testing of all images and test cases. If phases 1 and 2 have been effective there should not be any show-stopper bugs appearing at this stage. The broader testing is likely to reveal a large number of minor bugs though. These should be targeted at the next milestone or stable release updates. Track image validation testing at [http://iso.qa.ubuntu.com/ iso.qa.ubuntu.com].
Testing/ISO/Schedule (last edited 2009-09-29 15:22:39 by 63)