Ubuntu Wiki

To be carried out by: QA Team

Goals:

• Discover any major issues relating to an upgrade from the current stable release to the current development release
• Ensure that these issues are appropriately recorded and brought to the attention of the release team

Summary

The means to find upgrade problems are automatic testing, manual testing and user feedback.

The tests should cover:

• the ReleaseUpgrader (update-manager -d)

• CDROM upgrades (sh /cdrom/cdromupgrade)
• CDROM only upgrades (CDROM install without network, CDROM upgrade without network)
• manual upgrades (apt-get dist-upgrade)

Manual testing

This test covers: package upgrade issues, maintainer script failures, hardware regressions, and bugs in the ReleaseUpgrader.

Network upgrade

1. Perform a regular ubuntu install (preferably in a VM that can be snapshotted), or restore a snapshot to get a clean system state.

2. Run:

   $ update-manager -d 

3. check if the ReleaseUpgrade button shows up.

4. click on upgrade, check the release notes, check the text of the ReleaseUpgrader

5. wait for the dialog box that comes up and shows the calculated upgrade
6. do a sanity check over the changes
7. run the upgrade
8. report/fix any failures and repeat

CD upgrade

1. Install from CD without network access (or restore a snapshot)
2. Insert a CD and wait for update-notifier to come up with a dialog asking about the upgrade.
3. Check if the upgrade can be calculated, if the calculated upgrade makes sense and if it actually works.

Command-line upgrade

The apt-get dist-upgrade test uses a fresh install too. Then the sources.list is modified and apt-get dist-upgrade is run. Check if ubuntu-desktop is cleanly upgraded and if the output looks sane, if not. Check with apt-get dist-upgrade -o Debug::pkgProblemResolver=true -o Debug::pkgDepCache::AutoInstall=true why not. Report/fix any problems here and then run the actual upgrade. Check for failures and report/fix them.

Automatic testing

The AutoDistUpgradeTestingSpec describes the plans for feisty. We will have a automatic test in the datacenter that will test {ubuntu,kubuntu,edubuntu,xubuntu}-desktop upgrades, server upgrades and most-of-main upgrades. Failures will be sent to the test-failure mailing list. The test will be run in a chroot initially (maybe later in some form of VM).

This test covers: package upgrade issues, override problems, certain maintainer script failures, certain dependency issues.

User feedback

We should encourage users to test the upgrade. The spec scalable-installation-testing covers what needs to be done here.

Reporting upgrade problems

All upgrade problems should be tagged with $dist-upgrade and/or $dist-regression, targeted to the distribution release and monitored by the release team as high priority items that need to be resolved before the release.

CategoryProcess