• Test plan for component: chromium

• Upstream project: https://www.chromium.org/Home

• Ubuntu package URL (LP): https://launchpad.net/ubuntu/+source/chromium-browser

• Ubuntu packaging team (LP): https://launchpad.net/~chromium-team

• Ubuntu packaging repository: https://code.launchpad.net/~chromium-team/chromium-browser

Description

This is a test plan for chromium as used by Ubuntu Desktop.

Initial set up

• If testing the snap, install
  • the latest Ubuntu Desktop development release
  • the Chromium snap from the candidate channel

    • Don't use parallel install for the test, otherwise some steps will fail

  • xdg-desktop-portal-gnome (if you had to install it, make sure it's running with ps ax|grep xdg-desktop-portal-gnom[e], simplest way to ensure that is logging out and in)

• If testing the deb, install
  • Ubuntu Desktop Bionic

  • Verify that autopkgtests all pass for amd64, i386, arm64 and armhf

  • chromium-browser from the PPA

Ensure that only one virtual console is running a display server. It is common otherwise that dialog windows or even programs will open in the other display server.

Manual Tests

About

• Browse to about:version and verify that the version number, user-agent string and profile directory are as expected

  • If the user agent is purposedly changed (besides version number), verify that Google services still work (start a meeting on https://meet.google.com or check Google Drive).

• Browse to about:gpu and check the status of GPU support (in particular, if running on real hardware, as opposed to inside a virtual machine, Canvas, WebGL and WebGL2 should be hardware-accelerated)

Custom URL schemes handling

• Verify that clicking mailto:foobar@nospam.org opens your default e-mail client and starts composing a message

• Browse to https://snapcraft.io/0ad, click the green 'Install' button, then click the 'View in Desktop store' button, and verify that it opens the software store on the details page for 0 A.D.

Downloads

• Download a tarball, e.g. grace_5.1.25.orig.tar.gz. Open the downloads view with Ctrl+J and verify that the file is fully downloaded, that you can display its parent folder (this should open nautilus or your default file manager) and that you can open the downloaded file (should open in file-roller)

• Download a sample doc. Verify that the file is fully downloaded, that you can display its parent folder and that you can open the downloaded file

• Download a sample pdf, which should open and render a PDF file in the current tab. Verify that you can scroll through the pages, zoom/unzoom, and save (download) the file to disk

Default browser

• Open the system settings on the apps panel and verify that chromium is not the default browser (by default it is firefox)
• Browse to about:settings/defaultBrowser and click the button to make chromium the default browser. A dialog asking for confirmation will appear, allow it and then switch to the settings app, change panels and back to the default applications panel, and verify that now chromium is the default browser
• In the settings app, change the default web browser back to firefox, then refresh the about:settings/defaultBrowser tab in chromium and verify that the button to make chromium the default browser is visible again
• In the settings app, change the default web browser to chromium, then refresh the about:settings/defaultBrowser tab in chromium and verify that the button to make chromium the default browser is hidden

Theme

• Use GTK_THEME to try to launch Chromium with a local theme in $HOME/.local/share/themes. It must fail to load that theme and instead fall back to the default.

Referral

• Type any word in the omnibox and press enter to search it with the default search engine. Verify that the referral code has been added to the search URI:

  • with Google: &client=ubuntu-chr

  • with DuckDuckGo: &t=canonical

Translation

Visit a web page in a foreign language (e.g., German or Romanian). Verify that a page translation box pops up, suggesting translation from the correct language to your locale's one. Clicking your language should translate the page to it.

Rendering

• Browse to https://html5test.co and verify that the browser was correctly detected and that the score is as expected (currently 574/594 for chromium 123)

• Browse to https://browserleaks.com/geo and verify that you get prompted to allow/deny geolocation access. Allow it and verify that your approximate location is found

• Browse to http://webglsamples.org/aquarium/aquarium.html and verify that, on real hardware, you get a good FPS (60) with the default number of sprites (500 fishes)

Safe Browsing

• Browse to https://testsafebrowsing.appspot.com and click one of the links, verify that you get a dissuasive warning

Video playback

• Watch any 360° video on Youtube (for instance https://www.youtube.com/watch?v=wczdECcwRw0) and verify that you can pan with the mouse to move around in the scene while it's playing back. Verify that you can pause/unpause, enter/exit fullscreen, and that while the video is playing the screensaver or screen blanking is inhibited.

WebRTC

• Browse to https://mozilla.github.io/webrtc-landing/gum_test.html and verify that camera, microphone, and screen capture are working as expected.

  • Make sure to test full screen sharing under Wayland, it has been broken before.

Password storage

• Make sure "offer to save passwords" is enabled in settings.

Snap

Only perform test for new major releases.

[Test 1] With the password-manager-service interface disconnected (default),

--> MAIN <--

• Browse to https://yari-demos.prod.mdn.mozit.cloud/en-US/docs/Web/HTML/Element/input/password/_sample_.A_simple_password_input.html, input any word in the password field, and verify there's a little key icon in the URL bar, next to the bookmark (star) icon. Click the key, input a user name, and press Enter.

• Browse to about:password-manager/passwords that there is an entry for yari-demos.prod.mdn.mozit.cloud with the corresponding username and password (obfuscated by default).

--> END MAIN <--

[Test 2] Kill Chromium and connect the password-manager-service interface.

• Open Seahorse.
  • Lock the keyring.
• Open Chromium.
  • Verify that you get prompted for the keyring password; Enter the password.
  • Perform MAIN above.

Deb

• Perform MAIN above.
• Open seahorse (the default system keyring application), and under the Login keyring verify that there's an entry called "Chromium Safe Storage".

Widevine CDM

• Download the latest google chrome from https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb, install it with dpkg -i, restart chromium and verify that you can load and play back widevine encrypted content at https://shaka-player-demo.appspot.com/demo/ (for instance "Sintel")

Chromecast

• On real hardware, connect to a google account and watch any video on youtube. Use the burger menu to start casting it to an existing chromecast device that's connected to e.g. a TV and verify that the video plays there (reference: bug #1621753)