* Test plan for component: scopes
Component Checklist: <backlink to your component checklist>
Trunk URLs:
- lp:unity-scopes-api
- lp:unity-scopes-shell
Ubuntu Package URLs (LP):
Test Plan
This test plan is not supposed to be complete; use it to guide your manual testing so they dont miss big functional areas that are part in the component; also this should be used as guideline to inspire the exploratory testing which should be adapted smartly based on the real content of a MP
- Install on phone. During the execution of the tests, keep an eye on /home/phablet/.cache/upstart/scopes-registry.log for any exceptions or errors.
- Ensure you have some music and videos copied onto your phone
- Test the "apps" scope:
- Test first-use experience of My Music and Videos/My Videos scopes
- verify a large "Get started.." card is displayed in My Music scope when no local music is present on the phone.
- verify a large "Get started.." card is displayed in My Videos scope when no local videos are present on the phone.
- verify that camera app icon is displayed in the 1st category in Videos scope, and tapping it launches camera app.
- verify that uploading some music or videos (or recording a video with camera app) makes "Get started.." card disappear from respective scopes (and if any local video files are present, camera app icon disappears from Videos aggregator scope).
- Test scope settings:
- Go to the music scope
- Verify that your music artists, 7digital, and Grooveshark results appear.
- Search for music, and ensure appropriate results are returned (from local, 7digital, Grooveshark)
- Turn off 7digital and Grooveshark results in turn, and check searches only show from the enabled sources.
- Don't forget to re-enable all the sources again.
- Test the media scopes:
- Test the overview:
- Swipe from the bottom to see the overview.
- Pick a non-favourited scope, e.g. Amazon, and open it.
- Add the scope as a favourite (drop down at the top right).
- Go back to the home screen and check the scope is now swipable to (should be at the far right).
Install a new scope (e.g. YouTube scope) using the Ubuntu Store scope, check that it appears in the overview ("Manage Dash") page.
- Uninstall a scope using Ubuntu Store, check that it disappears from the overview.
- Test remote scopes:
- Ensure that network is not available (disconnect or enable "Flight mode").
- Remove /home/phablet/.cache/unity-scopes/remote-scopes.json file (if exists) and reboot the phone.
- Swipe from the bottom to see the overview, then navigate to the "All" tab.
- Verify that the Amazon scope is *not* visible in the list of scopes.
- Swipe down the network indicator, connect to network, then swipe the indicator menu back up again.
- After about 10-20s, the list of "All" scopes should refresh.
- Verify now that the Amazon scope *is* visible in the list of scopes.
- Open the Amazon scope by tapping on the icon.
- Ensure that some surfacing results appear in the scope.
- Search for something and ensure that the appropriate results are returned.
- Open a result and ensure that the preview looks correct (i.e. Appropriate image(s), texts, and a working "View on Amazon" button)
- Enable 'Flight mode' and restart the phone
- Swipe from the bottom to see the overview, then navigate to the "All" tab.
- Verify that you are in 'Flight mode' and remote scopes such as Amazon are still listed in the Manage view (note: click them however will not display any results while in Flight mode).
- Test online accounts access:
- Log-in from scope:
Install the YouTube scope using the Ubuntu Store scope (If not already installed).
Open the YouTube scope from the Manage Dash page.
When the scope has finished surfacing, check that the first result is a "Log-in to YouTube" button.
Tap the "Log-in to YouTube" result.
- A Google account log-in page should pop up.
- Select “Cancel”, and check that focus is simply returned to the scope with no further activity (the scope should not refresh).
Tap the "Log-in to YouTube" result again.
- Log in to your Google account, then tap "Accept" on the next page that appears.
- Once focus is returned to the scope, the scope should then refresh its results.
When the scope has finished refreshing, check that there is no longer a "Log-in to YouTube" button in the results.
- Enable / disable scope access:
Navigate to System Settings -> Accounts -> Google.
On the Google account page you should see an entry for "YouTube" under "Access to this account:". It should be enabled (green toggle switch).
Disable "YouTube" by tapping the green toggle switch next to the entry (slides left and turns grey).
Return to the YouTube scope.
- Pull down to refresh the results.
When the scope has finished surfacing, check that the "Log-in to YouTube" item is back.
Tap the "Log-in to YouTube" result, this should pop up a prompt to either "Allow", "Add another account...", or "Don't allow".
- Select "Don't allow", and check that focus is simply returned to the scope with no further activity (the scope should not refresh).
Tap the "Log-in to YouTube" result again, and this time select "Allow".
- Now check that focus is returned to the scope and the scope then refreshes its results.
When the scope has finished refreshing, check that there is no longer a "Log-in to YouTube" button in the results.
- Log-in from scope:
- Test pre-compiled Go scopes
Ensure that you have no Fitbit accounts configured under System Settings -> Accounts.
- Install the Fitbit scope using the Ubuntu Store scope (If not already installed).
- Open the Fitbit scope from the overview ("Manage Dash") page.
- When the scope has finished surfacing, you should see only one "Add your Fitbit account" result.
- Tap the "Add your Fitbit account" result.
- A Fitbit account log-in page should pop up.
- Log in to a Fitbit account.
- Once focus is returned to the scope the scope should then refresh its results.
- When the scope has finished refreshing, you should now see some categories (E.g. "My Activity", "My Food") and results under each.
- Open a result and ensure that the preview looks correct (i.e. Appropriate image(s), texts)
- Test location access (trusted prompt)
NOTE: this is a new functionality still in the silo as of 2016-05-31. With these changes Dash runs confined and access to location results in a trusted prompt shortly after first boot and a global ON/OFF toggle for the Dash (all scopes) in the Security & Privacy > Location pane of System settings. This global location toggle overrides any local scope settings ('Enable location data' checkbox of scope's settings; if Dash is denied access via apparmor, then individual location toggles in scope settings are hidden).
- Known bugs:
reboot on location access on arale: https://bugs.launchpad.net/ubuntu/+source/webbrowser-app/+bug/1512128
location is not available to scopes and apps after the wizard on first boot, requires rebooting: https://bugs.launchpad.net/ubuntu/+source/location-service/+bug/1415029
a reboot is required for change to take effect if global location access is changed from ON to OFF, i.e. denying location after it was allowed (this bug also affects apps, but doesn't require reboot, restarting an app is enough): https://bugs.launchpad.net/trust-store/+bug/1547573
suspected bug in the Qt sensors API https://bugs.launchpad.net/qtubuntu-sensors/+bug/1589517 which results in a race & location access denial on first request during Dash startup (e.g. "no location data vailable" in Nearby scope), requiring manual refresh.
the location hints displayed by some scopes are a bit misleading - https://bugs.launchpad.net/ubuntu-ux/+bug/1592719
- changing global location access for the Dash from OFF to ON may require a while before location gets available and scopes need manual refresh (pulling down to refresh) to pick new location.
to repeat the tests and simulate "first boot experience" (force the default state & location prompt) for testing purposes, remove /home/phablet/.local/share/UbuntuLocationService/trust.db (WARNING! This removes location settings for all apps!) and /home/phablet/.config/.scopesLocationPrompt and reboot.
- Reset the device to factory defaults and complete the wizard making sure you set up WIFI network. Verify a trusted prompt asking for location access for Scopes is displayed on first user-initiated search (start typing a search string or pull down to refresh with a scope which needs location).
- Select "Don't Allow". Reboot the device to workaround the bugs listed above, affecting first boot.
- Go to any scope which needs location, check that 'Enable location' checkbox is not displayed in scope settings pane. Verify that Nearby scope doesn't know the location after refreshing.
Go to System Settings > Security & Privacy > Location, verify "Scopes" app is available. Enable location access.
Go to a scope which needs location (such as NearBy) and refresh it.
- Go to the settings pane of that scope, verify 'Enable location data' option is now available and switch it ON in case it's disabled.
- Verify location is now available to the scope.
- Reboot the phone. Verify trust prompt for location access for Scopes is not shown anymore.
- Check that location used by scopes looks sensible and accurate.