TestingForTranslations

Summary

Most of the ISO testing we currently do does not contain any testing for installations or configurations in other languages but English. This specification tries to solve this identifying the different test cases that we need to cover.

Release Note

  • http://testcases.qa.ubuntu.com/Install will contain new testcases that will cover installations of Ubuntu in other languages but English.

  • Some of those tests will be added to the ISO tracker to be part of the ISO milestone testing.

Rationale

As the Ubuntu user spectrum is wider, we need to make sure that the same level of quality in installations in English and any other language are similar. We have been hitting bugs where a missing lang package made a non-English installation appear in English for some applications.

User stories

  • Marco wants to use his Ubuntu in Italian and helps testing Ubuntu milestones. He can install Ubuntu in Italian and report back his findings in the ISO tracker.

  • The Ubuntu Catalonia LoCo Team wants to help testing Ubuntu in Catalan as part of their local jams. They gather together for milestone testing, test the installation of Ubuntu in Catalan, and report issues in Launchpad and the ISO tracker.

Assumptions

  • Local people want to help testing Ubuntu in their native language.

Design

Testcases description

Testcases will be described in the Testcase Wiki, under /Install. They will follow the current testcases writing conventions.

They have to include:

Live CD

  • Languages included in the CD (Bengali (bn), German (de), English (en), Spanish (es), French(fr), Portuguese (pt), Xhosa (xh) for Karmic)
    • With network connection
      • Full language support should be installed
    • No network connection
      • Pop-up on reboot (when installed) saying your language support is not complete
  • Languages not included in the CD
    • Network connection - Same thing
    • No network connection
      • Pop-up on reboot (when installed) saying your language support is not complete (but no lang packs has been installed)
  • Try Live CD in English and install then in another language (it was broken once)
  • Keyboard layout and input method
    • CJK IM should be enabled by default

      • -> environment variables:

        • GTK_IM_MODULE=ibus
        • QT_IM_MODULE=ibus
        • XMODIFIERS=@im=ibus
    • Other languages - depending on layout chosen at install time (probably cannot be tested on the live CD)
  • Regional settings
    • The settings are in the locale variables
    • If we want to fully test this, tha applications using these settings should also be tested (e.g. gnome-panel, calendar)
  • Completion of translations for important applications
    • A Mago script could be written to test all the menus and see if they are translated

DVD

  • Full language support for any language, without the need of a full network connection

BoF agenda and discussion

Testing for translations

  • We currently do no have translations tests for ISO testing.
  • This is specially important while testing the DVD, where the language support
    • should be complete.
  • We think it would be helpful to add some installations tests that include
    • translation issues for Lucid to add those kind of cases to the ISO tracker.
  • Current installation test cases at:
  • Suggestion: it would be interesting to have some advice from the testing team
    • on:
    • Testing/finding out untranslatable strings
    • Testing language packs on stable releases
  • Encourage non-english speaking developers or advance users to use the
    • translated version of Ubuntu

https://wiki.ubuntu.com/Specs/TestingForTranslations

Test case proposals

  • Non-English installation from a LiveCD
    • Without Internet connection
      • Are all strings on the installer translated? (to be checked during the
        • installation process)
      • Is a notification displayed mentioning that language support is not
        • complete? (Only for those languages without language packs in the CD)
    • With Internet connection
      • Are all strings on the installer translated? (to be checked during the
        • installation process)
      • Is the language pack corresponding to the initial language selection
        • automatically downloaded and installed? Check it is listed on

          System > Preferences > Language Support (Ubuntu) or K menu > Favorites > System Settings > Country/Region and language

      • Do high visible applications appear completely translated? Visually
        • inspect them by starting and using them.
          • Firefox
          • Nautilus (Ubuntu)
          • GNOME menus (Ubuntu)
          • Dolphin (Kubuntu)
          • Kickoff (Kubuntu)
          • ...
  • Non-English installation from a LiveCD, using the "Install Ubuntu" link from
    • within the Live session
    • Same tests as above

Note: the best languages to use for testing are Spanish, French, Brazilian Portuguese and Italian, since they are usually very complete and it's easy to spot untranslatable vs. untranslated strings in them. We should test other non-Latin languages as well though, perhaps Arabic as an RTL one with good support and Simplified Chinese.

Timo's recipe for generic non-English "first install" testing in VirtualBox:

1. Disable network support, install non-English Ubuntu without network connection. * Check through all dialogs in the installer and see there are no non-translated strings ** If non-translated strings, check ubiquity templates if the string is there - if it is, probably non-issue and will be fixed * After installation, shutdown virtual machine and re-enable network support * Start Ubuntu to the desktop and *wait what happens*. Ubuntu should automatically report about missing translations support. ** 1. Follow the dialog and accept refreshing repositories and install suggested missing support packages. 2. Go again to the language-selector after installation is complete, if it still suggests something it's a bug and should have been already installed during the first run. ** If no automatic popup appears, file a bug.

2. Enable network support, install non-English Ubuntu with network connected via Network Manager if necessary (for wired network NM connects automatically on the live CD) * Start Ubuntu to the desktop, it should be translated. * Go to language-selector, it should optimally not suggest anything since everything should have been fetched during the installation.

(3. Using either installation, look through all the GNOME menus * No English should be observed in any menu entry. If there is, do something about it. It does require knowledge about whether the bug is in the source package, ie. desktop entry not translatable via Rosetta. If it's translatable, then it's translators' job.)

* The installer should be completed translated no matter the language

  • Test weeks start on Mondays before milestone releases (which are on Thursdays)

Live CD

* Languages included in the CD (Bengali (bn), German (de), English (en), Spanish (es), French (fr), Portuguese (pt), Xhosa (xh) for Karmic)

  • Network connection
    • Full language support should be installed
  • No network connection
    • Pop-up on reboot (when installed) saying your language support is not complete

* Languages not included in the CD

  • Network connection - Same thing
  • No network connection
    • Pop-up on reboot (when installed) saying your language support is not complete (but no lang packs has been installed)

* Try Live CD in English and install then in another language (it was broken once)

* Keyboard layout and input method

  • CJK IM should be enabled by default

    • -> environment variables:

      • GTK_IM_MODULE=ibus
      • QT_IM_MODULE=ibus
      • XMODIFIERS=@im=ibus
  • Other languages - depending on layout chosen at install time (probably cannot be tested on the live CD)

* Regional settings

  • The settings are in the locale variables
  • If we want to fully test this, tha applications using these settings should also be tested (e.g. gnome-panel, calendar)

* Completion of translations for important applications

  • A Mago script could be written to test all the menus and see if they are translated

Alternate CD

DVD

* Full language support for any language

ACTIONS

* Look for packages per language in the Live CD (adi) * Write the test cases in the testcase wiki (ara) * Review testcases (David - to communicate to translations team) * Get a list of packages that should be installed (Arne) * Check language support on the DVD (Arne) * Check the language support on the alternate CD (David) * David to announce to the translations community to see if anyone comes up with more useful test cases

Packages installed (via Arne)

1. Language-packs For each language we have 6 possible language packs: # For all installations: language-pack-$LCODE-base language-pack-$LCODE # Only for Gnome: language-pack-gnome-$LCODE-base language-pack-gnome-$LCODE # Only for KDE: language-pack-kde-$LCODE-base language-pack-kde-$LCODE

The -base packages contain the full set of translations at release time. The other packages contain the updates, released after release. At release time the update packages are empty.

2. Language-support For each language there are up to three meta packages, which contain dependencies onto other packages. The list is specified in langpack-o-matic, which builds those meta-packages and is attached in the language-support-depends.tar.bz2 tarball (one text file per language code). There are three categories of dependencies:

  • in: input methods
  • fn: fonts
  • wa: writing assistance

The text files contain these prefixes to tell langpack-o-matic which meta package should contain the dependency.

The three meta packages are:

  • language-support-input-$LCODE
  • language-support-fonts-$LCODE
  • language-support-writing-$LCODE

Each of these meta packages may or may not be installed, the user can decide to install them selectively in language-selector. Further more there is a general language-support-$LCODE meta package, which just depends on all of those three meta packages.

At system install time, ubiquity just pulls the general language-support-$LCODE package for the target language. Therefor all those dependencies should be installed on a freshly installed system.

3. Conditional packages The file pkg_depends, taken from language-selector, lists packages, which only get installed when a given dependency package is already installed. There are two categories:

  • tr: translations
  • wa: writing assistance

The format of the file is explained in the file itself.

The language-support-common package contains a command line tool 'check-language-support', which checks if all packages for a given language, all languages or all languages installed on the system are installed and lists the missing ones. This tool only checks for the conditional packages and language-packs, though (cases 1 and 3). See the man page for how to use it. Smile :)

Files:

DVD Language support (via Arne)

I have downloaded the manifest from the daily DVD builds for Lucid and looked over it. My gut feeling says that language support on the DVD is complete.

If you can help out with that, that would be very much appreciated.

The check is fairly simple, it just needs to be coded.

The main list of packages is 'language-pack-*' and 'language-support-*' from the archive and their dependencies. Further more, we have stuff, which is pulled in on a case by case basis, i.e. thunderbird-locale-* only gets pulled when thunderbird is actually installed on the user's system. The list for those is:

gnome-user-guide-* openoffice.org-l10n-* openoffice.org-help-* thunderbird-locale-* enigmail-locale-* lightning-extension-locale-* sunbird-locale-* sword-language-pack-* gimp-help-* evolution-documentation-* kde-l10n-* kde-i18n-* koffice-i18n-* mozvoikko openoffice.org-voikko

CategorySpec

Specs/TestingForTranslations (last edited 2010-02-17 13:59:30 by 172)