Schedule

Differences between revisions 22 and 42 (spanning 20 versions)
Revision 22 as of 2007-03-19 16:49:36
Size: 4355
Editor: cF5A6BF51
Comment:
Revision 42 as of 2009-09-29 15:22:39
Size: 3023
Editor: 63
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Community ISO testing schedule = <<Include(Testing/Header)>>
Line 3: Line 3:
== Ubuntu Desktop CD ==
[[BR]]
|| '''Installation method''' || '''i386''' [https://bugs.launchpad.net/ubuntu-iso-tests/+bug/93120 http://people.ubuntu.com/~henrik/testing/inc/info.png]|| '''amd64''' [https://bugs.launchpad.net/ubuntu-iso-tests/+bug/93121 http://people.ubuntu.com/~henrik/testing/inc/info.png] ||
|| check CD || '''everyone''' || '''everyone''' ||
|| live session || '''everyone''' || '''everyone''' ||
|| erase disk || '''[https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~alauppe alauppe]''' [[BR]] ''[https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~bryceharrington bryce], [https://launchpad.net/~michl Michl], [https://launchpad.net/~jt-jtholmes jtholmes], [[BR]] [https://launchpad.net/~dg10050 dg10050], [https://launchpad.net/~deathter elrohir], dlr_pe'' || '''[https://launchpad.net/~darroch Darroch] [https://launchpad.net/~jjblock JBlock]''' [[BR]] ||
|| manual partitioning || '''[https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~bryceharrington bryce], [https://launchpad.net/~huygens-25 huygens], [[BR]] [https://launchpad.net/~rubenr-canada rubenr-canada], [https://launchpad.net/~jt-jtholmes jtholmes], [https://launchpad.net/~deathter elrohir]''' [[BR]] ''[https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~jwcampbell j1mc]'' || '''[https://launchpad.net/~adelgado1313 adelgado]''', ''[https://launchpad.net/~jjblock JBlock]''[[BR]] ||
|| auto-resize || '''[https://launchpad.net/~michl Michl], [https://launchpad.net/~dg10050 dg10050],''' [[BR]] ''[https://launchpad.net/~mixium mixium], [https://launchpad.net/~alauppe alauppe], [https://launchpad.net/~chillisoup chillisoup], [https://launchpad.net/~sno-squee mrsno]'' || '''[https://launchpad.net/~sno-squee mrsno]''' ||
|| WinFOSS || '''[https://launchpad.net/~stgraber stgraber], [https://launchpad.net/~mixium mixium], [https://launchpad.net/~linuturk Linuturk], [https://launchpad.net/~michl Michl]''' || '''[https://launchpad.net/~adelgado1313 adelgado], [https://launchpad.net/~jjblock JBlock]''', ''[https://launchpad.net/~sno-squee mrsno], [https://launchpad.net/~darroch Darroch]'' ||
||<tablestyle="float:right; font-size: 0.9em; background:#F1F1ED; margin: 0 0 0 0;" style="padding:0.5em;"><<TableOfContents(2)>>||
Line 13: Line 5:
== Kubuntu Desktop CD ==
[[BR]]
|| '''Installation method''' || '''i386''' [https://bugs.launchpad.net/ubuntu-iso-tests/+bug/93120 http://people.ubuntu.com/~henrik/testing/inc/info.png]|| '''amd64''' [https://bugs.launchpad.net/ubuntu-iso-tests/+bug/93121 http://people.ubuntu.com/~henrik/testing/inc/info.png] ||
|| check CD || '''everyone''' || ||
|| live session || '''everyone''' || ||
|| erase disk || '''[https://launchpad.net/~firephoto firephoto], [https://launchpad.net/~mib-mibgames mibuk], [https://launchpad.net/~abattoir abattoir], dlr_pe, [https://launchpad.net/~damontimm thornomad]''' [[BR]] || [[BR]] ''[https://launchpad.net/~darroch Darroch]'' ||
|| manual partitioning || '''[https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~jwcampbell j1mc], [https://launchpad.net/~huygens-25 huygens]''' [[BR]] ''[https://launchpad.net/~vorian Vorian], [https://launchpad.net/~firephoto firephoto], [https://launchpad.net/~adelgado1313 adelgado], [https://launchpad.net/~pbcartwright pbcartwright]'' || '''[https://launchpad.net/~gianni.net leibowitz]''' ||
|| auto-resize || '''[https://launchpad.net/~vorian Vorian], [https://launchpad.net/~sno-squee mrsno], [https://launchpad.net/~chillisoup chillisoup], [https://launchpad.net/~pbcartwright pbcartwright]''' [[BR]] ''[https://launchpad.net/~firephoto firephoto], [https://launchpad.net/~dg10050 dg10050], [https://launchpad.net/~abattoir abattoir]'' || '''[https://launchpad.net/~gianni.net leibowitz], [https://launchpad.net/~sno-squee mrsno]''' ||
|| WinFOSS || '''[https://launchpad.net/~bjkemp nailBnny], [https://launchpad.net/~sno-squee mrsno]''' || '''[https://launchpad.net/~gianni.net leibowitz]''', ''[https://launchpad.net/~sno-squee mrsno], [https://launchpad.net/~darroch Darroch]'' ||
Testing of the major milestones Beta, RC and Final stretches over about a week and is done in two stages, smoke and validation testing.
Line 23: Line 7:
''Secondary install tests in italics'' = Karmic ISO test schedule =
Line 25: Line 9:
'''Note:''' ''Please do not add yourself to this page directly, but email iso.testing AT gmail.com (read by Henrik and Emilio) so we can coordinate. '' ||<rowbgcolor="#cccccc"> '''Release''' || '''Task''' || '''Date''' ||
||<rowbgcolor="#FFFFCC"> Beta || Smoke testing || Sept. 24-29 ||
||<rowbgcolor="#FFFFCC"> || ISO validation testing || Sept. 29-Sept. 30 ||
||<rowbgcolor="#FFEBBB"> Release Candidate || Smoke testing || Oct. 15-20 ||
||<rowbgcolor="#FFEBBB"> || ISO validation testing || Oct. 20-22||
||<rowbgcolor="#FFCCCC"> Final || Smoke testing || Oct. 22-27 ||
||<rowbgcolor="#FFCCCC"> || ISO validation testing || Oct. 27-28 ||
Line 27: Line 17:
----
Return to [:/..:Community Testing]
= Testing details =

In the first ''smoke test'' phase we perform a wide series of tests to cover every ISO and install method but do not attempt to test every combination. The aim is to find bugs quickly so they can be addressed for the milestone. As candidate ISO images become available we test each image following the relevant test case to each ISO in order to ensure that every ISO we ship works as expected. This phase is quite labor intensive and will require the participation of 20 or more people in a short window of time.

== Smoke testing ==

The first round of testing is designed to find high impact bugs in a short period of time so they can be addressed before the milestone is released. Smoke testing should occur just after the archives are frozen. Upgrade and netboot testing can start immediately; ISO testing starts after the first daily images based on the frozen archive appear. The following elements should receive coverage during smoke testing:

 * Core distro flavours: '''Ubuntu, Kubuntu, Edubuntu, Xubuntu, UNR, UMR, Server'''
 * Supported architectures: '''i386, amd64, sparc, armel'''
 * Image types: '''Live CD, Alternate, Netboot''' (not DVDs)
 * Install methods: '''erase disk, auto-resize, manual partitioning, entire disk with encryption, Netboot, Upgrade'''
 * Select features: '''Session management applet, encrypted folder, Wubi'''

Testing will be coordinated in '''#ubuntu-testing''' and the results tracked on a '''[[Testing/ISO/Smoke|simple checklist]]'''. The number of tests is kept low intentionally, but they should be performed with care (e.g. there are only 3 Live CD session test in total, but they should be thorough; testing much of the default applications and installing others).

== Image validation testing ==

Complete validation testing of all images and test cases. If the smoke testing was effective there should not be any show-stopper bugs appearing at this stage. The broader testing is likely to reveal a large number of minor bugs though. These should be targeted at the next milestone or stable release updates. Track image validation testing at '''[[http://iso.qa.ubuntu.com/|iso.qa.ubuntu.com]]'''.

Testing of the major milestones Beta, RC and Final stretches over about a week and is done in two stages, smoke and validation testing.

Karmic ISO test schedule

Release

Task

Date

Beta

Smoke testing

Sept. 24-29

ISO validation testing

Sept. 29-Sept. 30

Release Candidate

Smoke testing

Oct. 15-20

ISO validation testing

Oct. 20-22

Final

Smoke testing

Oct. 22-27

ISO validation testing

Oct. 27-28

Testing details

In the first smoke test phase we perform a wide series of tests to cover every ISO and install method but do not attempt to test every combination. The aim is to find bugs quickly so they can be addressed for the milestone. As candidate ISO images become available we test each image following the relevant test case to each ISO in order to ensure that every ISO we ship works as expected. This phase is quite labor intensive and will require the participation of 20 or more people in a short window of time.

Smoke testing

The first round of testing is designed to find high impact bugs in a short period of time so they can be addressed before the milestone is released. Smoke testing should occur just after the archives are frozen. Upgrade and netboot testing can start immediately; ISO testing starts after the first daily images based on the frozen archive appear. The following elements should receive coverage during smoke testing:

  • Core distro flavours: Ubuntu, Kubuntu, Edubuntu, Xubuntu, UNR, UMR, Server

  • Supported architectures: i386, amd64, sparc, armel

  • Image types: Live CD, Alternate, Netboot (not DVDs)

  • Install methods: erase disk, auto-resize, manual partitioning, entire disk with encryption, Netboot, Upgrade

  • Select features: Session management applet, encrypted folder, Wubi

Testing will be coordinated in #ubuntu-testing and the results tracked on a simple checklist. The number of tests is kept low intentionally, but they should be performed with care (e.g. there are only 3 Live CD session test in total, but they should be thorough; testing much of the default applications and installing others).

Image validation testing

Complete validation testing of all images and test cases. If the smoke testing was effective there should not be any show-stopper bugs appearing at this stage. The broader testing is likely to reveal a large number of minor bugs though. These should be targeted at the next milestone or stable release updates. Track image validation testing at iso.qa.ubuntu.com.

Testing/ISO/Schedule (last edited 2009-09-29 15:22:39 by 63)