Ubuntu Server Testing

Overview

Areas under test and opportunities for testing of Ubuntu Server. At the end of this document is a list of opportunities.

Contents

  1. Ubuntu Server Testing
    1. Overview
    2. ISOs and Cloud Images
      1. Automated Testing
      2. Manual Testing
    3. Packages
    4. Bug Work
      1. Daily Triage
      2. Backlog Triage
      3. Backlog Work
    5. Opportunities

ISOs and Cloud Images

As the delivery vehicle for obtaining Ubuntu Server the ISOs and cloud images must be tested. This is the first touch users have with Ubuntu Server and therefore a positive user experience is essential. ISOs are produced on two release cadences:

Automated Testing

Testing of the daily ISOs is done in an automated fashion whenever a new "pending" daily ISO is produced. This automated testing consists of the following:

  1. Download and verify hashes
  2. Smoke test of a single, default install to verify no obvious issues
  3. Promote "pending" to "current"
  4. Installs with a variety of preseed and tasksel options set to validate specific software packages and configuration settings.

This testing occurs on Jenkins instance using jobs created with Jenkins job builder and the test cases used to verify the preseed and tasksel.

Manual Testing

Because of the importance of the ISO experience manual testing is still required. This testing occurs via the ISO Testing Tracker.

Testing is outlined by point-release milestone. Inside each milestone has a list of the architectures to test and inside each architecture is a list of test case. Above each test case there is a link to the specific ISO that needs to be tested.

You can register with the iso testing tracker and subscribe to the ubuntu server testcases so that you'll be notified whenever a new ubuntu-server iso needs to be tested.

Packages

The server team's list of owned packages need testing.

The first and foremost option is to create and utilize dep8 tests. Writing these tests help verify standard functionality and that the packages are usable. Many packages do not have tests written at all and some could use even more tests.

Other options for work include writing Verification Tests.

Bug Work

As a part of owning quality it is important to stay involved in reported defects. During the defect resolution process, bugs should have tests verifying that the bug is actually solved. These tests should then be put back into the package's tests (e.g. dep8) or pushed back to the upstream package's project.

Daily Triage

There is an effort by the Server Team to deal with server specific bugs.

Backlog Triage

Triage consists of a rotating team member who is in charge of sorting through the list of new bugs. This consists of checking that enough information to go work on the bug and that it is a legitimate and valid problem in Ubuntu Server to solve. If there is not enough information or the issue is not a bug, they work with the reporter to either gather additional information or close the bug.

Backlog Work

Once a bug is validated and known to be reproducible it is added to the backlog of bugs. These bugs will have at the very least steps to reproduce and are ready of work. They also may be very simple as they already have a patch that needs to be accepted. In either case, these bugs are ready for work.

Opportunities

If you have questions about anything documented here, please feel free to ask in the #ubuntu-server channel on freenode.

Above everything, one of the best ways to work on quality in Ubuntu Server is to use it! Try setting up Ubuntu Server on a cloud, run it as a KVM host you remotely connect to, use it as a development server you deploy to, use it for work.

Here are some other ideas in getting involved:

Testing/Server (last edited 2017-01-26 18:23:44 by powersj)