ImproveServerISOTestcases

Summary

While testing Server ISOs our test cases are pretty sparse. We should brainstorm some more ideas to improve the quality of our server ISO testing.

Rationale

In an effort to find bugs during our ISO testing we should improved our server ISO testcases and increase the number of tests that we perform.

User stories

  • Charles is an Ubuntu developer who wants to test to see if he could connect his windows machne to his Ubuntu machine when he is testing the server ISO. Charles was able to do so after following instructions in the ISO test tracker.
  • Josh is a community member who wishes to help test the ISO, Josh selects the Kickstart installation and follows the ISO test tracker to completion.

Assumptions

The QA team will be driving this specification with input from the Server team.

Design

With the help of the server team, the current ISO tracker instructions for the server ISO will be updated and will feature 4 new test cases. Additional, the existing iso test cases will be integrated into checkbox as well.

File Server Testing

The testcase will be updated to test the client as well as the server. THis will include ubuntu <=> ubuntu and windows <=> ubuntu.

LAMP Stack

The testcase will be updated to test that PHP functionality will be included as well. This might be accomplished by doing a simple php script such as:

<? phpinfo(); ?>

Additional tests

The following tests will be added as well:

  1. No network interface install
  2. No internet connection install
  3. Kickseed install (kickstart)
  4. Repair system option

UI Changes

No specific UI changes are expected to the ISO test tracker.

Code Changes

Checkbox will be updated to include the Server ISO testcases.

Test/Demo Plan

After the updated and new cases have been written, we will have the QA team go through the tests and make sure that we get the results that we desire.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

Topic:

 - wondering if all test cases should fall under ISO testing. Would it make sense to do testing outside of ISO testing period and how would this be formalized. There are multiple common uses cases that could be tested independently of ISO testing before release, such as:
 - tomcat + java application test
 - samba accessibility
 - active directory integration
 - etc...
 
 
Expand the testcases to cover server components/integration testing:
 - File server testing:
   * samba testing (client):
     - integration testing: windows <-> ubuntu
   * samba testing (server)
 - LAMP task: run php scripts.
 - Mail task: send email and read it

Use the iso tracker to leverage the community outside the milestone period. Use it for manual testing.

New test cases to add 
 - No network interface install
 - No internet connection install
 - Multiple network interface install
 - cr3: kickseed install (kickstart)
 - repair system option

== ACTIONS == 
Server team to write up the test cases.
QA team to run the new test cases.
cr3 to integrate existing iso test cases in checkbox: http://testcases.qa.ubuntu.com/Install/ServerWhole


CategorySpec

Specs/ImproveServerISOTestcases (last edited 2009-12-14 10:47:46 by 14)