scopes

* Test plan for component: scopes

Test Plan

This test plan is not supposed to be complete; use it to guide your manual testing so they dont miss big functional areas that are part in the component; also this should be used as guideline to inspire the exploratory testing which should be adapted smartly based on the real content of a MP

  • Install on phone. During the execution of the tests, keep an eye on /home/phablet/.cache/upstart/scopes-registry.log for any exceptions or errors.
  • Ensure you have some music and videos copied onto your phone
  • Test the "apps" scope:
  • Test first-use experience of My Music and Videos/My Videos scopes
    • verify a large "Get started.." card is displayed in My Music scope when no local music is present on the phone.
    • verify a large "Get started.." card is displayed in My Videos scope when no local videos are present on the phone.
    • verify that camera app icon is displayed in the 1st category in Videos scope, and tapping it launches camera app.
    • verify that uploading some music or videos (or recording a video with camera app) makes "Get started.." card disappear from respective scopes (and if any local video files are present, camera app icon disappears from Videos aggregator scope).
  • Test scope settings:
    • Go to the music scope
    • Verify that your music artists, 7digital, and Grooveshark results appear.
    • Search for music, and ensure appropriate results are returned (from local, 7digital, Grooveshark)
    • Turn off 7digital and Grooveshark results in turn, and check searches only show from the enabled sources.
    • Don't forget to re-enable all the sources again.
  • Test the media scopes:
  • Test the overview:
    • Swipe from the bottom to see the overview.
    • Pick a non-favourited scope, e.g. Amazon, and open it.
    • Add the scope as a favourite (drop down at the top right).
    • Go back to the home screen and check the scope is now swipable to (should be at the far right).
    • Install a new scope (e.g. YouTube scope) using the Ubuntu Store scope, check that it appears in the overview ("Manage Dash") page.

    • Uninstall a scope using Ubuntu Store, check that it disappears from the overview.
  • Test remote scopes:
    • Ensure that network is not available (disconnect or enable "Flight mode").
    • Remove /home/phablet/.cache/unity-scopes/remote-scopes.json file (if exists) and reboot the phone.
    • Swipe from the bottom to see the overview, then navigate to the "All" tab.
    • Verify that the Amazon scope is *not* visible in the list of scopes.
    • Swipe down the network indicator, connect to network, then swipe the indicator menu back up again.
    • After about 10-20s, the list of "All" scopes should refresh.
    • Verify now that the Amazon scope *is* visible in the list of scopes.
    • Open the Amazon scope by tapping on the icon.
    • Ensure that some surfacing results appear in the scope.
    • Search for something and ensure that the appropriate results are returned.
    • Open a result and ensure that the preview looks correct (i.e. Appropriate image(s), texts, and a working "View on Amazon" button)
    • Enable 'Flight mode' and restart the phone
    • Swipe from the bottom to see the overview, then navigate to the "All" tab.
    • Verify that you are in 'Flight mode' and remote scopes such as Amazon are still listed in the Manage view (note: click them however will not display any results while in Flight mode).
  • Test online accounts access:
    • Log-in from scope:
      • Install the YouTube scope using the Ubuntu Store scope (If not already installed).

      • Open the YouTube scope from the Manage Dash page.

      • When the scope has finished surfacing, check that the first result is a "Log-in to YouTube" button.

      • Tap the "Log-in to YouTube" result.

      • A Google account log-in page should pop up.
      • Select “Cancel”, and check that focus is simply returned to the scope with no further activity (the scope should not refresh).
      • Tap the "Log-in to YouTube" result again.

      • Log in to your Google account, then tap "Accept" on the next page that appears.
      • Once focus is returned to the scope, the scope should then refresh its results.
      • When the scope has finished refreshing, check that there is no longer a "Log-in to YouTube" button in the results.

    • Enable / disable scope access:
      • Navigate to System Settings -> Accounts -> Google.

      • On the Google account page you should see an entry for "YouTube" under "Access to this account:". It should be enabled (green toggle switch).

      • Disable "YouTube" by tapping the green toggle switch next to the entry (slides left and turns grey).

      • Return to the YouTube scope.

      • Pull down to refresh the results.
      • When the scope has finished surfacing, check that the "Log-in to YouTube" item is back.

      • Tap the "Log-in to YouTube" result, this should pop up a prompt to either "Allow", "Add another account...", or "Don't allow".

      • Select "Don't allow", and check that focus is simply returned to the scope with no further activity (the scope should not refresh).
      • Tap the "Log-in to YouTube" result again, and this time select "Allow".

      • Now check that focus is returned to the scope and the scope then refreshes its results.
      • When the scope has finished refreshing, check that there is no longer a "Log-in to YouTube" button in the results.

  • Test pre-compiled Go scopes
    • Ensure that you have no Fitbit accounts configured under System Settings -> Accounts.

    • Install the Fitbit scope using the Ubuntu Store scope (If not already installed).
    • Open the Fitbit scope from the overview ("Manage Dash") page.
    • When the scope has finished surfacing, you should see only one "Add your Fitbit account" result.
    • Tap the "Add your Fitbit account" result.
    • A Fitbit account log-in page should pop up.
    • Log in to a Fitbit account.
    • Once focus is returned to the scope the scope should then refresh its results.
    • When the scope has finished refreshing, you should now see some categories (E.g. "My Activity", "My Food") and results under each.
    • Open a result and ensure that the preview looks correct (i.e. Appropriate image(s), texts)
  • Test location access (trusted prompt)
    • NOTE: this is a new functionality still in the silo as of 2016-05-31. With these changes Dash runs confined and access to location results in a trusted prompt shortly after first boot and a global ON/OFF toggle for the Dash (all scopes) in the Security & Privacy > Location pane of System settings. This global location toggle overrides any local scope settings ('Enable location data' checkbox of scope's settings; if Dash is denied access via apparmor, then individual location toggles in scope settings are hidden).

    • Known bugs: Remarks:
      • changing global location access for the Dash from OFF to ON may require a while before location gets available and scopes need manual refresh (pulling down to refresh) to pick new location.
      • to repeat the tests and simulate "first boot experience" (force the default state & location prompt) for testing purposes, remove /home/phablet/.local/share/UbuntuLocationService/trust.db (WARNING! This removes location settings for all apps!) and /home/phablet/.config/.scopesLocationPrompt and reboot.

      Test steps:
      • Reset the device to factory defaults and complete the wizard making sure you set up WIFI network. Verify a trusted prompt asking for location access for Scopes is displayed on first user-initiated search (start typing a search string or pull down to refresh with a scope which needs location).
      • Select "Don't Allow". Reboot the device to workaround the bugs listed above, affecting first boot.
      • Go to any scope which needs location, check that 'Enable location' checkbox is not displayed in scope settings pane. Verify that Nearby scope doesn't know the location after refreshing.
      • Go to System Settings > Security & Privacy > Location, verify "Scopes" app is available. Enable location access.

      • Go to a scope which needs location (such as NearBy) and refresh it.

      • Go to the settings pane of that scope, verify 'Enable location data' option is now available and switch it ON in case it's disabled.
      • Verify location is now available to the scope.
      • Reboot the phone. Verify trust prompt for location access for Scopes is not shown anymore.
      • Check that location used by scopes looks sensible and accurate.

Process/Merges/TestPlan/scopes (last edited 2016-06-24 14:26:25 by stolowski)