InstallationTestCases

Introduction

This specification should outline both test cases for installation completion but also for expected failures such as low memory and under size HDD scenarios where the installation is not a possibility. This specification only outlines test cases concerned with the installation completing successfully, and not logical functionality of the installer itself, which is covered in the state transition testcases.

Approval

Author

Dan Chapman

dpniel

Ubuntu Member

Approved By

<name>

< lp profile >

< position >

Related Documents

Ref #

Document Name

Link

01

Installer Design - Ubuntu Installation Process

http://goo.gl/H12x6V

02

Test Analysis and Specification for Ubiquity

Automated testing for Ubiquity

03

Ubiquity Test Plan

Ubiquity Test Plan

04

State Transition Test Cases

../StateTransitionTestCases

Contents

  1. Introduction
  2. Approval
  3. Related Documents
  4. Considerations
  5. Installation Test Cases
    1. Install Scenario 1 - Less Than System Requirements
    2. Install Scenario 2 - Default Installation
    3. Install Scenario 3 - LVM installation
    4. Install Scenario 4 - LVM with Encryption Installation
    5. Install Scenario 5 - Custom/Manual Partition installation
    6. Install Scenario 6 - Auto-Resize Installation
      1. a) Using Default
      2. b) Using LVM
      3. c) Using LVM with Encryption
    7. Install Scenario 7 - Manual-Resize installation
    8. Install Scenario 8 - Erase & Re-Install
      1. a) Using Default
      2. b) Using LVM
      3. c) Using LVM with Encryption
    9. Install Scenario 9 - Upgrade Installation
    10. Install Scenario 10 - Re-Install Re-Using Home Installation
    11. Install Scenario 11 - OEM Installation
    12. Install Scenario 12 - Free Software Only Installation

Considerations

There are many factors that need to be taken into account for creation of these test cases,

Installation Type

Default options
LVM install
LVM with encryption
Custom partitioning
Empty HDD
Resize/other OS installed
No swap
Auto-resize
Upgrade
Erase and Re-install
Re-Install saving user data
OEM
Free Software only

Connectivity

Ethernet
Wireless
No Network

Memory

Less Than Min Req
Min Req
Greater Than Min Req

Hard Drive Size

Less Than Min Req
Min Req
Greater Than Min Req

Power Source

Connected
Battery

Localisation

Different language choices

File System Format

ext2
ext3
ext4
JFS
XFS
ReiserFS
btrfs
fat 16
fat 32
Swap
Encrypted partition
Empty partition

The combination of all these factors results in 1000’s of possible test cases, so we need to decide on a subset of scenarios.

In the next section I propose some 168 initial test case scenarios we should try and aim to achieve. A lot of the test cases are similar and will only need one or two arguments changing in the actual autopilot test case or the test case runner, so although there does seem a lot, in reality it won’t require too much extra effort to reach this initial goal.

Installation Test Cases

Test case names are in the format of

“InstallScenario-InstalledOS-Connectivity-TestCaseNumber”

Installed OS codes are as follows

F

*buntu flavor

W

Windows OS

M

Mac OS

X

No OS / Empty HDD

L

Other linux distro

And Connectivity,

Eth

Ethernet

WF

Wi-Fi

NN

No Network

The different connectivity options will only be used on test cases where we have an empty HDD, to test every scenario against each connection type is not very practical and basically triples our test case amount for every OS already installed on the HDD.

Install Scenario 1 - Less Than System Requirements

We should test that we cannot install with less than the system requirements.

Test Cases

Details

IS1-X-Eth-TC1

less than required memory
Greater than or equal required HDD size

IS1-X-Eth-TC2

less than required memory
less than required HDD size

IS1-X-Eth-TC3

greater than or equal required memory
less than required HDD size

Install Scenario 2 - Default Installation

Test Cases

Details

IS2-X-Eth-TC1
IS2-X-WF-TC2
IS2-X-NN-TC3

English install
Minimum Memory Requirements
Minimum HDD Requirements

IS2-X-Eth-TC4
IS2-X-WF-TC5
IS2-X-NN-TC6

Non-English install
Minimum Memory Requirements
Minimum HDD Requirements

IS2-X-Eth-TC7
IS2-X-WF-TC8
IS2-X-NN-TC9

English install
> Minimum Memory Requirements
> Minimum HDD Requirements

IS2-X-Eth-TC10
IS2-X-WF-TC11
IS2-X-NN-TC12

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 3 - LVM installation

Test Cases

Details

IS3-X-Eth-TC1
IS3-X-WF-TC2
IS3-X-NN-TC3

English install
Minimum Memory Requirements
Minimum HDD Requirements

IS3-X-Eth-TC4
IS3-X-WF-TC5
IS3-X-NN-TC6

Non-English install
Minimum Memory Requirements
Minimum HDD Requirements

IS3-X-Eth-TC7
IS3-X-WF-TC8
IS3-X-NN-TC9

English install
> Minimum Memory Requirements
> Minimum HDD Requirements

IS3-X-Eth-TC10
IS3-X-WF-TC11
IS3-X-NN-TC12

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 4 - LVM with Encryption Installation

Test Cases

Details

IS4-X-Eth-TC1
IS4-X-WF-TC2
IS4-X-NN-TC3

English install
Minimum Memory Requirements
Minimum HDD Requirements

IS4-X-Eth-TC4
IS4-X-WF-TC5
IS4-X-NN-TC6

Non-English install
Minimum Memory Requirements
Minimum HDD Requirements

IS4-X-Eth-TC7
IS4-X-WF-TC8
IS4-X-NN-TC9

English install
> Minimum Memory Requirements
> Minimum HDD Requirements

IS4-X-Eth-TC10
IS4-X-WF-TC11
IS4-X-NN-TC12

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 5 - Custom/Manual Partition installation

With there being a mass amount of partitioning layout and file system formats we should try a subset of layouts and each FS type at least once, now i’m not sure that they should all be tested at the same time or continue with how the current tests are implemented. Which currently selects a random predefined configuration during each test run, this way at some point all the FS types will get tested and we have tried various layouts. The current configurations can be seen here, the majority of them naively assume we always have the default 12GB of space to play with. It would ideally be better to have seperate configurations for the minimum system requirements test cases, which the test cases below assume this to be true.

This has been difficult to implement in non-english languages with autopilot due to relying on label values to navigate this section. But there are test cases for it to be implemented in the future should it become easier

Test Cases

Details

IS5-X-Eth-TC1
IS5-X-WF-TC2
IS5-X-NN-TC3

English install
Minimum Memory Requirements
Minimum HDD Requirements
Random Config

IS5-X-Eth-TC4
IS5-X-WF-TC5
IS5-X-NN-TC6

Non-English install
Minimum Memory Requirements
Minimum HDD Requirements
Random Config

IS5-X-Eth-TC7
IS5-X-WF-TC8
IS5-X-NN-TC9

English install
> Minimum Memory Requirements
> Minimum HDD Requirements
Random Config

IS5-X-Eth-TC10
IS5-X-WF-TC11
IS5-X-NN-TC12

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements
Random Config

Install Scenario 6 - Auto-Resize Installation

a) Using Default

Test Cases

Details

IS6a-F-Eth-TC1
IS6a-W-Eth-TC2
IS6a-M-Eth-TC3
IS6a-L-Eth-TC4

English install
Use default install options
Minimum Memory Requirements
Minimum HDD Requirements

IS6a-F-Eth-TC5
IS6a-W-Eth-TC6
IS6a-M-Eth-TC7
IS6a-L-Eth-TC8

Non-English install
Use default install options
Minimum Memory Requirements
Minimum HDD Requirements

IS6a-F-Eth-TC9
IS6a-W-Eth-TC10
IS6a-M-Eth-TC11
IS6a-L-Eth-TC12

English install
Use default install options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS6a-F-Eth-TC13
IS6a-W-Eth-TC14
IS6a-M-Eth-TC15
IS6a-L-Eth-TC16

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements

b) Using LVM

Test Cases

Details

IS6b-F-Eth-TC1
IS6b-W-Eth-TC2
IS6b-M-Eth-TC3
IS6b-L-Eth-TC4

English install
Use LVM install options
Minimum Memory Requirements
Minimum HDD Requirements

IS6b-F-Eth-TC5
IS6b-W-Eth-TC6
IS6b-M-Eth-TC7
IS6b-L-Eth-TC8

Non-English install
Use LVM install options
Minimum Memory Requirements
Minimum HDD Requirements

IS6b-F-Eth-TC9
IS6b-W-Eth-TC10
IS6b-M-Eth-TC11
IS6b-L-Eth-TC12

English install
Use LVM install options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS6b-F-Eth-TC13
IS6b-W-Eth-TC14
IS6b-M-Eth-TC15
IS6b-L-Eth-TC16

Non-English install
Use LVM install options
> Minimum Memory Requirements
> Minimum HDD Requirements

c) Using LVM with Encryption

Test Cases

Details

IS6b-F-Eth-TC1
IS6b-W-Eth-TC2
IS6b-M-Eth-TC3
IS6b-L-Eth-TC4

English install
Using LVM Encryption options
Minimum Memory Requirements
Minimum HDD Requirements

IS6b-F-Eth-TC5
IS6b-W-Eth-TC6
IS6b-M-Eth-TC7
IS6b-L-Eth-TC8

Non-English install
Using LVM Encryption options
Minimum Memory Requirements
Minimum HDD Requirements

IS6b-F-Eth-TC9
IS6b-W-Eth-TC10
IS6b-M-Eth-TC11
IS6b-L-Eth-TC12

English install
Using LVM Encryption options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS6b-F-Eth-TC13
IS6b-W-Eth-TC14
IS6b-M-Eth-TC15
IS6b-L-Eth-TC16

Non-English install
Using LVM Encryption options
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 7 - Manual-Resize installation

We can use the same configs used in the custom partition page tests, we just need to resize to make enough space first.

Test Cases

Details

IS7-F-Eth-TC1
IS7-W-Eth-TC2
IS7-M-Eth-TC3
IS7-L-Eth-TC4

English install
Minimum Memory Requirements
Minimum HDD Requirements
Random Config

IS7-F-Eth-TC5
IS7-W-Eth-TC6
IS7-M-Eth-TC7
IS7-L-Eth-TC8

Non-English install
Minimum Memory Requirements
Minimum HDD Requirements
Random Config

IS7-F-Eth-TC9
IS7-W-Eth-TC10
IS7-M-Eth-TC11
IS7-L-Eth-TC12

English install
> Minimum Memory Requirements
> Minimum HDD Requirements
Random Config

IS7-F-Eth-TC13
IS7-W-Eth-TC14
IS7-M-Eth-TC15
IS7-L-Eth-TC16

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements
Random Config

Install Scenario 8 - Erase & Re-Install

a) Using Default

Test Cases

Details

IS8a-F-Eth-TC1
IS8a-W-Eth-TC2
IS8a-M-Eth-TC3
IS8a-L-Eth-TC4

English install
Use default install options
Minimum Memory Requirements
Minimum HDD Requirements

IS8a-F-Eth-TC5
IS8a-W-Eth-TC6
IS8a-M-Eth-TC7
IS8a-L-Eth-TC8

Non-English install
Use default install options
Minimum Memory Requirements
Minimum HDD Requirements

IS8a-F-Eth-TC9
IS8a-W-Eth-TC10
IS8a-M-Eth-TC11
IS8a-L-Eth-TC12

English install
Use default install options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS8a-F-Eth-TC13
IS8a-W-Eth-TC14
IS8a-M-Eth-TC15
IS8a-L-Eth-TC16

Non-English install
> Minimum Memory Requirements
> Minimum HDD Requirements

b) Using LVM

Test Cases

Details

IS8b-F-Eth-TC1
IS8b-W-Eth-TC2
IS8b-M-Eth-TC3
IS8b-L-Eth-TC4

English install
Use LVM install options
Minimum Memory Requirements
Minimum HDD Requirements

IS8b-F-Eth-TC5
IS8b-W-Eth-TC6
IS8b-M-Eth-TC7
IS8b-L-Eth-TC8

Non-English install
Use LVM install options
Minimum Memory Requirements
Minimum HDD Requirements

IS8b-F-Eth-TC9
IS8b-W-Eth-TC10
IS8b-M-Eth-TC11
IS8b-L-Eth-TC12

English install
Use LVM install options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS8b-F-Eth-TC13
IS8b-W-Eth-TC14
IS8b-M-Eth-TC15
IS8b-L-Eth-TC16

Non-English install
Use LVM install options
> Minimum Memory Requirements
> Minimum HDD Requirements

c) Using LVM with Encryption

Test Cases

Details

IS8c-F-Eth-TC1
IS8c-W-Eth-TC2
IS8c-M-Eth-TC3
IS8c-L-Eth-TC4

English install
Using LVM Encryption options
Minimum Memory Requirements
Minimum HDD Requirements

IS8c-F-Eth-TC5
IS8c-W-Eth-TC6
IS8c-M-Eth-TC7
IS8c-L-Eth-TC8

Non-English install
Using LVM Encryption options
Minimum Memory Requirements
Minimum HDD Requirements

IS8c-F-Eth-TC9
IS8c-W-Eth-TC10
IS8c-M-Eth-TC11
IS8c-L-Eth-TC12

English install
Using LVM Encryption options
> Minimum Memory Requirements
> Minimum HDD Requirements

IS8c-F-Eth-TC13
IS8c-W-Eth-TC14
IS8c-M-Eth-TC15
IS8c-L-Eth-TC16

Non-English install
Using LVM Encryption options
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 9 - Upgrade Installation

Test Cases

Details

IS9-F-Eth-TC1

English upgrade
Minimum Memory Requirements
Minimum HDD Requirements

IS9-F-Eth-TC2

Non-English upgrade
Minimum Memory Requirements
Minimum HDD Requirements

IS9-F-Eth-TC3

English upgrade
> Memory Requirements
> Minimum HDD Requirements

IS9-F-Eth-TC4

Non-English upgrade
> Minimum Memory Requirements
> Minimum HDD Requirements

Install Scenario 10 - Re-Install Re-Using Home Installation

Todo at later date

Install Scenario 11 - OEM Installation

Todo at later date

Install Scenario 12 - Free Software Only Installation

Todo at later date

QATeam/AutomatedTesting/InstallationTestCases (last edited 2014-01-03 17:37:57 by 89-168-6-168)