Description

This is a test plan for gnome-software as used by Ubuntu Desktop. It covers the standard features of Gnome Software as well as the support for snap packages.

Initial set up

Install latest Ubuntu Desktop development release
Install gnome-software from (PPA ppa:ubuntu-desktop/gnome-software ? -proposed)

$ sudo apt-add-repository ppa:ubuntu-desktop/gnome-software
$ sudo apt update
$ sudo apt upgrade
$ killall gnome-software

Then start gnome-software

Or if the package is in proposed Enable Proposed instead of installing from the PPA.

Manual Tests

Test Plan Summary

General functionalities
- Search
- Navigation
‘All apps’ screen functionalities
‘Installed apps’ screen functionalities
- List of apps actually installed
‘Updates’ screen functionalities
- Update package cache
- Install all updates
‘App details’ functionalities
- Sections of the screen are properly filed
- Install
- Remove
- Open
- Reviews
Snap specific cases, on desktop, open gnome-software:
- The tests cover the most basic features for snapd search, login to the store, install and remove

Test Sets

General Functionalities

Search

Show / Hide search (skip for Xenial - this search entry is always shown in gnome-software 3.20)
1. Click on the magnifying glass
  1. Verify that the search field is displayed
2. Click on the magnifying glass again
  1. Verify that the search field is hidden

For all the following test cases, the search field must be displayed. Besides they assume that the package cache is up to date. Between each test, click on the magnifying glass twice to hide/show the search field and reset the search.

Existing application packaged as deb (installed)
1. Enter the string 'gnome-software'
  1. Verify that the result contains only 1 line with gnome-software.
  2. Verify that the icon of the application is displayed properly (not a 'missing' icon)
  3. Verify that the rating of the application is displayed (dark and light grey stars)
  4. Verify that the description is displayed and matched the description of gnome-software.
  5. Verify that 'installed' is displayed.
  6. Verify that clicking on the line shows the detailed description of the application.
Description of the application as a deb is translated
1. Switch the language of the desktop environment to 'French'
2. Enter the string 'gnome-software'
  1. Verify that the description is in French
Existing application packaged as deb (not installed)
1. Pre-requisite: The application 'entangle' must not be installed
2. Enter the string 'entangle'
  1. Verify that the result contains only 1 line with the application.
  2. Verify that the icon of the application is displayed properly (not a 'missing' icon)
  3. Verify that the rating of the application is displayed (dark and light grey stars)
  4. Verify that the description is displayed and matched the description of the application.
  5. Verify that 'installed' is not displayed.
  6. Verify that clicking on the line shows the detailed description of the application.
Existing application packaged as snap (installed)
1. Pre-requisite: The application 'hello-unity' must not be installed
2. Enter the string 'hello-unity'
  1. Verify that the result contains only 1 line with the application.
  2. Verify that the icon of the application is displayed properly (not a 'missing' icon)
  3. Verify that the rating (if any) of the application is displayed (dark and light grey stars)
  4. Verify that the description is displayed and matched the description of gnome-software.
  5. Verify that 'installed' is displayed.
  6. Verify that clicking on the line shows the detailed description of the application.
Existing application packaged as snap (not installed)
1. Pre-requisite: The application 'hugo' must not be installed
2. Enter the string 'hugo'
  1. Verify that the result contains only 1 line with the application.
  2. Verify that the icon of the application is displayed properly (not a 'missing' icon)
  3. Verify that the rating of the application is displayed (dark and light grey stars)
  4. Verify that the description is displayed and matched the description of the application.
  5. Verify that 'installed' is not displayed.
  6. Verify that clicking on the line shows the detailed description of the application.
Mixed results (deb, snap, installed and not installed)
1. Search for 'krita'
  1. Verify that the list contains 2 results: 1 line for the snap package and 1 line for the debian package.
No match
1. Search for 'non-existent'
  1. Verify that the list is empty and the message 'No match found' is displayed.
Number of characters in search string
1. Type slowly in the search field 'ge'
  1. Verify that no search is performed
2. Type 'd'
  1. Verify that a search is performed and the application 'gedit' is in the list of results
3. Append 'it' in the search field
  1. Verify that the search is narrowed down and the number of results is smaller.
Multi words search
1. In the search field type "gnome software"
  1. Verify that 'gnome-software' is in the list of results.
Special characters
1. I could not find a good example of package. Maybe this test is not useful
Several searches in a row
1. Type 'gimp' in the search field
  1. Verify that 'gimp' is in the list of results
2. Type 'editor' in the search field
  1. Verify that the result contains several record matching the search string.
Search relevance
1. TODO: Not sure what the heuristic is here

Tab bar
1. Click on 'All'
  1. Verify that the 'All' section is diplayed
  2. Verify that the action on the upper right is 'Search' (magnifying glass)
2. Click on 'Installed'
  1. Verify that the 'Installed' section is displayed
  2. Verify that the action on the upper right is 'Select' (check mark)
3. Click on 'Updates'
  1. Verify that the 'Updates' section is displayed
  2. Verify that the action on the upper right is 'Update all'.
Menu 'Software & updates'
1. Click on Software menu and select 'Software & Updates'
  1. Verify that software properties opens
Menu 'About'
1. Click on Software menu and select 'About'
  1. Verify that the about dialog opens.
Menu 'Quit'
1. Click on Software menu and select 'Quit'
  1. Verify that the application exits.

All Apps

Page Sections
1. Check the content of the page
  1. Verify that the following sections are displayed in that order
    1. Featured Application
    2. Categories
    3. Editor's pick
2. Verify that none of them is empty.
Featured application
1. Check the featured application
  1. Verify that an image is displayed to illustrate the application
2. Click on the image
  1. Verify that it opens the detailed page of the application
- Categories
  1. Check the section
    1. Verify that 6 categories are displayed when the section is folded
  2. Click on the arrow below the categories
    1. Verify that the section ufolds
    2. Verify that 9 categories are displayed
  3. Click on 'Audio & Video'
    1. Verify that the 'Audio & Video' category is displayed and All is selected
    2. Verify that the subcategories are displayed
    3. Verify that the applications listed match the category
  4. Click on back (top left button) and click on 'Games'
    1. Repeat previous verifications with 'Games'
Editor's Picks
1. Check the Editor's Picks section.
  1. Verify that several applications (packaged as snap or deb) are displayed
  2. Verify that a check mark is displayed on the top right corner of the card for installed card and nothing otherwise
  3. Verify that all the applications have an icon
  4. Verify that ratings are displayed for applications with ratings (eg gedit)
2. Click on the first application
  1. Verify that the detailed page of the application is displayed
3. Click on back and click on the second application
  1. Verify that the detailed page of the application is displayed

Installed

Pre-requisite: You must be an administrator of the system under test to perform most of these tests.

Content of the page
1. Check the content of the page
  1. Verify that only installed applications are displayed.
  2. Verify that the list contains debian packages and snap packages (use dpkg -l and snap list from the command line to double-check)
  3. Verify that a remove button is displayed on the right of the screen.
2. Scroll down to the bottom of the list
  1. Verify that and Add-ons subsection is displayed
Uninstall a debian package
1. Pre-requisites:
  - Install 'krita' on the system
2. Identify 'krita' in the list
3. Click on 'Remove'
4. A confirmation dialog is displayed
5. Click on Remove to confirm the removal
6. A password dialog is displayed to grant admin privileges, enter your password and authenticate
  1. Verify that the removal is performed
  2. Verify that the application is not on the list anymore
  3. Open a terminal and verify with dpkg that the application is uninstalled
An uninstalled application (deb) can be reinstalled
1. Go to the 'All' tab
2. Click on 'Search'
3. Enter 'krita'
  1. Verify that the list shows 2 results: One for the debian package and one for the snap package
  2. Verify that the debian package is not marked as 'Installed'
4. Click on the line of the debian package to open the detailed page
5. Click on 'Install'
  1. After a successful installation, verify that the application is installed with dpkg
6. Go back then to the 'Installed' tab
  1. Find 'krita' in the list of installed applications
Uninstall a snap package
1. Pre-requisites:
  - Install the snap version of 'openscad-plars' on the system
2. Identify 'openscad-plars' in the list
3. Click on 'Remove'
4. A confirmation dialog is displayed
5. Click on Remove to confirm the removal
6. A password dialog is displayed to sign in to the Snap Store, enter your password and authenticate
  1. Verify that the removal is performed
  2. Verify that the application is not on the list anymore
  3. Open a terminal and verify with snap that the application is uninstalled
An uninstalled application (snap) can be reinstalled
1. Go to the 'All' tab
2. Click on 'Search'
3. Enter 'openscad'
  1. Verify that the list shows 2 results: One for the debian package and one for the snap package
  2. Verify that the debian package is not marked as 'Installed'
4. Click on the line of the snap package to open the detailed page
5. Click on 'Install'
  1. After a successful installation, verify that the application is installed with snap
6. Go back then to the 'Installed' tab
  1. Find 'openscad' in the list of installed applications
Uninstall several packages (mix of deb and snap)
1. TBD: Removal of snap packages crash gs

Updates

At least one update available
1. Pre-requisite: At least one package must be updated
  1. Check that the number of packages to update is displayed on the name of the tab
2. Click on 'Install all updates'
  1. Check the the packages are updated.
Update packages cache - no updates
1. Pre-requisite: Apply all the updates on the system
  1. Check that there is no number of packages to install displayed on the name of the 'Updates' tab
2. Click on the 'refresh' icon (top left corner)
  1. Check that the package cache is refreshing
  2. Check that the message "Software is up to date" is displayed with the date of the last update
  3. Check that there is still no number of packages to install displayed on the name of the 'Updates' tab
3. Click on 'Update All'
  1. Check that this action has no effect

To verify that actions on apt triggered by gnome-software check the entries in the journal for org.debian.apt. In a terminal run the following command:

journalctl -torg.debian.apt

Package details

Content of the page
1. Pre-requisite: gnome-clocks must not be installed
2. Search for 'clock' and select 'Gnome Clocks'
  1. Verify that the page contains:
    1. A title and a short desctription
    2. Rating with stars
    3. One or several screenshots
    4. A detailed description
    5. Detailed ratings
    6. User's comments
Install a package
1. Pre-requisite: gnome-clocks must not be installed
2. Search for 'clock' and select 'Gnome Clocks'
3. Click on 'Install'
4. When the password dialog shows up enter your administrator's password
  1. Verify that the installation ends successfully
  2. Verify that the application is installed (use dpkg on the command line)
  3. Verify that the 'Install' button is replaced by 'Launch' and 'Remove' buttons
Launch an application
1. Pre-requisite: gnome-clocks must be installed
2. Go to the detailed page of the application
3. Click on 'Launch'
  1. Verify that the application starts
Remove a package
1. Pre-requisite: gnome-clocks must be installed
2. Go to the detailed page of the application
3. Click on 'Remove'
4. When the confirmation dialog shows up, confirm the removal
5. When the authentication dialog shows up enter the password of the administrator
  1. Verify that the application is removed (use dpkg on the command line)
  2. Verify that the 'Launch' and 'Remove' buttons are replaced by 'Install'
Install a 3rd party debian package
1. Visit https://www.google.com/chrome/ in Firefox and download Chrome.
2. Open with "Software Install"
  1. Verify that the Ubuntu/GNOME Software app starts and shows a basic page about Google Chrome
3. Click Install.
  1. Verify that Google Chrome installs

TODO

Add negative test cases
Add test cases for application add-ons (eg gnome-software plugin)
Add non-functional test cases (performance on startup, search, ...)

Process/Merges/TestPlans/gnome-software (last edited 2017-08-31 11:59:41 by jibel)