Qt Manual Testing

New major Qt version of all components

For new Qt 5.x releases see the complete landing plan at https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0AjuCdq68GSyVdFI4QzNQdWpfME5aMEV2VXo0cUpOMkE and instructions for updating phone or desktop at https://wiki.ubuntu.com/Touch/QtTesting -

• You can't use 'citrain' tool to upgrade!

• Do not forget to remove the QML cache before rebooting!

Also, the same tests as for patching qtbase/qtdeclarative/qtxmlpatterns.

On the desktop, larger testing of Qt 5 apps.

Checks to be done by certain upstreams

unity8: make sure "make qmluitests" passes (or at least fails the same than with the old Qt)

Patching a single Qt component or new upstream minor (stable) versions

qtbase, qtdeclarative, qtxmlpatterns

These are basic building blocks of Qt and need careful testing. The testing can be modified according to the actual change (for example if the code change affects only animations, it's useless to test everything). AP:s should be run always as they are a usefully automated way to catch issues.

Run all AP:s

Run all the same Autopilot tests as shown in the dashboard http://ci.ubuntu.com/smokeng/utopic/touch/ (there's a qt-gatekeeper jenkins job that can be used for doing this)

Run app manual tests

Run additional smoke-testing of apps by executing test plans found with "-app" search on the page https://wiki.ubuntu.com/Process/Merges/TestPlan/

Desktop smoke-testing

Test Ubuntu SDK (Qt Creator) and KDE Plasma 5.

Other modules (like qtpim, qtlocation)

Test manually reverse dependencies well enough, including AP:s of apps where applicable.

qtpim: see Process/Merges/TestPlan/qtpim

qtlocation: test OSMTouch

Special instructions

Testing networking code in Qt Base

Description

This is an additional test plan for QNetworkAccessManager, QNetworkSession and QNetworkRequest on Ubuntu Touch. It does not cover test cases for Qt on the desktop, or any unit tests that may be run.

Ubuntu One login error

Check bug #1564142. "When logging in, if you enter the wrong password, it should show you an error message stating that either the email or password was incorrect. However, a generic network error message is being shown instead."

Basic Tests

These tests need to be conducted with mobile data and wifi connections configured and working.

1. System settings check for Updates "hang".

1. Open system settings and click on Updates and then quickly switch wifi off or on

   • can be done either from ui or nmcli radio wifi on (or off). Although ui works more like user would.

   • update should either have connection error or show results
   • repeat this alternating wifi on/off
2. Switch wifi off or on and quickly click on Updates
3. Click on Updates and then quickly switch on airplane mode
4. Switch off airplane mode and hit Updates

2. Optionally, download bzr branch lp:~lorn-potter/+junk/52700 compile and run. This will test to see if network requests will recover from connection change.

• While command is running:
• Turn off wifi make sure requests become successful after a short amount of time.
• Turn on wifi and sure requests become successful after a short amount of time.
• In both instances, network errors should happen when there is no network connectivity.

3. Login to Ubuntu One and use wrong password, it should show you an error message stating that either the email or password was incorrect and not a generic network error message.

The following is only related to NM backend code changes, not normal upgrades.

It is hinted in the bug #1357321 comment #94 that the remaining problems with using the old generic plugin include Qt continuing to use the 3G connection for some data connections even when WiFi becomes available and connected. Dual SIM is also only properly supported in the new code.

The data being transferred can be for example checked with /sbin/ifconfig, taking note of the data transfer amounts so far and then eg. doing connection changes, and checking whether the 3G ones stays unchanged like it should or whether it continues to increase when eg reloading scopes and such.

Update 2014-12-08: 1. Suggested approach is get the device outside of wifi signal, then back in (or pulling the plug on wifi and putting it back in) 2. A separate test application offered at https://bugs.launchpad.net/savilerow/+bug/1357321/comments/99

The old instructions below may be useful in general.

Old instructions

These were the earlier suggested tests for testing the LP: #1357321 Network Manager backend fix. However, since a Network Manager workaround went in, the bug is not reproducable this way.

Reproducing without PPA:

1. Boot up with WiFi and 3G enabled, stay on Apps page (don't do anything)

2. Disable wifi (=switches to 3G).
3. adb shell + rm -rf ~/.cache/unity8-dash/
4. Switch to video scope and pull down to refresh. Excepted: refreshes, images load. Result: refreshes, text shown but images not.

Testing 3G connection functionality in general:

1. Use a different network settings and/or reboot
2. adb shell + rm -rf ~/.cache/unity8-dash/ # clean scope cache
3. adb shell + ping 8.8.8.8 # leave on running if you want to monitor whether 3G stays up
4. Pull down on video (or music or other scopes) to fresh and check image loading. Do any other network testing as wished.

Suggested network combinations to test:

1. 3G only, wifi "disabled"
2. 3G only, wifi "enabled" but no known access points
3. compare to wifi enabled and connected

Desktop smoke test

Smoke-test eg. Qt Creator on the desktop.