Automated package install/remove/upgrade testing

There are currently two system for automatic package testing:

• automatic package testing

  • https://wiki.ubuntu.com/AutomatedTestingDeployment

• automatic upgrade testing

  • http://people.ubuntu.com/~mvo/automatic-upgrade-testing/

Those have different goals, the package tester runs package self-tests and the upgrader tester checks if a upgrade from one given version and one given set of packages will success or not.

Automatic upgrade testing

The current automatic upgrade testing system works well but is limited because it does only run in a chroot and diverts some important scripts (like invoke-rc.d). This limitation should be fixed by using a xen or qemu setup to test the upgrade.

Additionally it should do a mass install/upgrade test: install as many package together as possible and make sure that each package is installed at least once (this requires more than a single install/upgrade run for this test).

Automated package testing

In addition to the current features of automatic package test it should perform a test like piuparts: check if the package installs cleanly, does it use prompting in a maintainer script, does it remove/purge cleanly.

Releated work

Debian summer of code: "Automated Upgrade Testing Using QEMU"

The conflicts checker http://people.ubuntu.com/~robertc/possible-conflicts/