Internationalisation

Differences between revisions 19 and 41 (spanning 22 versions)
Revision 19 as of 2009-10-15 11:58:35
Size: 18559
Editor: 69
Comment: Added translations category
Revision 41 as of 2010-08-18 14:21:17
Size: 2654
Editor: 140
Comment: Added new sections
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
= Translations in Ubuntu (from a developer perspective) = = Ubuntu Internationalization Guide =
Line 10: Line 10:
== Getting Started ==
 * '''Primer''': This [[/InternationalizationPrimer|internationalization primer]] is a great place to get started.
Line 12: Line 15:
||<tablestyle="width:50%;" style="width: 40px; background-color: #565248;text-align: center;">{{attachment:Translations/info.png}}||<style=" text-align: left;" >Make sure to read the TranslationLifecycle in the [[Translations|translations wiki pages]] for a general overview of how translations work in Ubuntu|| ''General information on the tools and processes related to the internationalization of Ubuntu''
Line 14: Line 17:
=== Glossary of i18n-related terms ===  * [[TranslationLifecycle|Overview of the translations lifecycle]]
 * [[/Glossary|Glossary of i18n-related terms]]
Line 16: Line 20:
'''gettext'''<<BR>> == Coding ==
Line 18: Line 22:
'''Translation domain''' (domain)<<BR>>
The translation domain is a unique string assigned by the programmer in the code (usually in the build system) and used by the gettext functions to locate the message catalog where translations will be loaded from. The general form to compute the catalog's location is:
''Information on writing or modifying internationalized applications for Ubuntu''
Line 21: Line 24:
{{{
dir_name/locale/LC_category/domain_name.mo
}}}

which in Ubuntu expands to either `/usr/share/locale/$LOCALE/LC_MESSAGES/domain_name.mo` for translations not included in language packs (not in `main`) or `/usr/share/locale-langpack/$LOCALE/LC_MESSAGES/domain_name.mo` for those shipped in language packs. `$LANGUAGE` is generally the [[http://en.wikipedia.org/wiki/ISO_639-1|ISO 639-1 2-letter code]] (e.g. 'ca' or 'de') or [[http://en.wikipedia.org/wiki/ISO_639-2|ISO 639-2 3-letter code]] (e.g. 'ast') for the particular language. As an example, when using Nautilus in a Catalan locale with Ubuntu, the gettext functions will look for the message catalogue at:

{{{
/usr/share/locale-langpack/ca/LC_MESSAGES/nautilus.mo
}}}

The corresponding translation template should have the same translation domain in its filename, e.g. `nautilus.pot`. The translation domain must be unique across all applications and packages. I.e. something generic like `messages.pot` won't work.

<<Anchor(TranslationTemplate)>>
'''Translation template''' (POT file, PO template file)<<BR>>

'''Translation''' (PO file)<<BR>>

<<Anchor(MessageCatalog)>>
'''Message catalog''' (MO file)<<BR>>

'''intltool'''<<BR>>

'''Imports queue'''<<BR>>
 * [[/Coding|Internationalization of source code]]
Line 47: Line 28:
<<Anchor(LanguagePacks)>>
=== Language Packs ===
''Information on packaging internationalized packages for Ubuntu''
Line 50: Line 30:
==== General ====
Packages in `main` and `restricted` don't contain [[#MessageCatalog|translations (*.mo) files]] themselves, they are stripped during the build on the Launchpad buildds and put into `language-pack-*` packages instead. This is done in an attempt to bundle the translations you're interested in and cut down the occupied space.
 * [[/Packaging|Internationalization of packages]]
Line 53: Line 32:
||<tablestyle="width:100%;" style="width: 40px; background-color: #565248;text-align: center;">{{attachment:Translations/info.png}}||<style=" text-align: left;" >Once a package is promoted to `main`, it only needs to be rebuilt the first time after promotion so that [[https://launchpad.net/ubuntu/+source/pkgbinarymangler|pkgbinarymangler]] can strip the translations. After this, import to Launchpad Translations and export in language packs "will just happen"|| == Documentation ==
Line 55: Line 34:
==== Translation templates ====
For `pkgstriptranslations` (in the `pkgbinarymangler` package) to do the job right and translations to be imported into Launchpad, please make sure that your package in `main` or `restricted` '''generates a [[#TranslationTemplate|POT translation template]] during the build'''. It does not necessarily need to be shipped in the source or in the binary package. Generating it during the build is good enough.
''Information on how to adapt the build system to include internationalization support''
Line 58: Line 36:
||<tablestyle="width:100%;" style="width: 40px; background-color: #565248;text-align: center;">{{attachment:Translations/warning.png}}||<style=" text-align: left;" >If your package does not generate a POT template on build, its translations will remain as ''Blocked'' in the [[https://translations.launchpad.net/+imports|Launchpad Translations import queue]] and will not be imported||  * [[/Docs|Internationalization of documentation]]
Line 60: Line 38:
This can be achieved by running the following in `debian/rules`: == Build system ==
Line 62: Line 40:
 * For packages using CDBS and automake:
  * You just need to include `/usr/share/cdbs/1/rules/langpack.mk` in `debian/rules`
  * If the package is a GNOME one, and you are using the `gnome.mk` CDBS class already, you should be good to go, since it already includes `langpack.mk`
  * KDE packages: TODO
  * If for some reason the POT template cannot be generated during the build, see the sections below.
''Information on how to adapt the build system to include internationalization support''
Line 68: Line 42:
 * Other packages, using `intltool` usually require:
  * A call to `intltool-update -p` in `debian/rules`, or
  * a call to an already existing Makefile target to build the POT (this does exist quite often in fact).

 * Packages which don't use `intltool`:
  * Need to get fixed individually
  * Usually by adding a call to `xgettext` and `po/POTFILES.in`, or
  * Finding an already existing Makefile target. e.g. Use `$(MAKE) -C po update-po` instead of `(cd po; make update-po; cd ..)`.

 * Packages which use `python-distutils-extra`:
  * Template creation should happen automagically :)
  * TODO: explanation of the command and auto modules

=== Other translation files ===
==== Desktop Entries ====

Translations of [[http://www.freedesktop.org/wiki/Specifications/desktop-entry-spec|desktop entries]] (.desktop files) are also stripped out and included in language packs.

Contrary to what other distros do (except currently OpenSUSE), the translations are not expected to be in the .desktop files themselves (static translations), but rather in the same .mo file of the application, from which they are then loaded at runtime. We do this in order to be able to translate them in Launchpad and to ship them in language packs, so that they can be edited and updated by Ubuntu translators.

The runtime loading is done by a [[https://launchpad.net/glib/+bug/3935|patch in glib]] (or in [[http://patches.ubuntu.com/k/kdelibs/extracted/kubuntu_56_langpacks_desktop_files.diff|kdelibs]] for Kubuntu), which recognises the the '''X-GNOME-Gettext-Domain''' [[http://standards.freedesktop.org/desktop-entry-spec/latest/ar01s08.html|additional key]] we add to all Ubuntu .desktop files and adds gettext support when loading them. Note that if the .desktop file already contains translations, these static translations will be preferred.

As an example for this key in action, here's how it is done in the Transmission source package in Jaunty (from the `20_add_X-Ubuntu-Gettext-Domain.diff` patch):

{{{
Index: transmission-1.40-2ubuntu1/gtk/transmission.desktop.in
===================================================================
--- transmission-1.40-2ubuntu1.orig/gtk/transmission.desktop.in 2008-12-09 18:57:13.000000000 +0000
+++ transmission-1.40-2ubuntu1/gtk/transmission.desktop.in 2008-12-09 18:58:54.000000000 +0000
@@ -9,3 +9,4 @@
 Type=Application
 MimeType=application/x-bittorrent;
 Categories=Network;FileTransfer;P2P;GTK;
+X-GNOME-Gettext-Domain=transmission
}}}

||<tablestyle="width:100%;" style="width: 40px; background-color: #565248;text-align: center;">{{attachment:Translations/info.png}}||<style=" text-align: left;" >If the package is using CDBS and the [[http://packages.ubuntu.com/search?searchon=contents&keywords=langpack.mk&mode=exactfilename&suite=karmic&arch=any|langpack.mk]] rule (or the [[http://packages.ubuntu.com/search?searchon=contents&keywords=gnome.mk&mode=exactfilename&suite=karmic&arch=any|gnome.mk]] class, which includes it) it will generally not be necessary to use a patch for this, as `langpack.mk` will take care of it||

Also note that `X-GNOME-Gettext-Domain` used to be called `X-Ubuntu-Gettext-Domain` in the past.

In summary, for packages in `main` we strip the translations out of the .desktop files, and glib/kdelibs carry a distro patch to load translations from the .mo file. They basically use the `X-Ubuntu-Gettext-Domain` key to figure out where to get the translation from.

Additional information:

 * [[LangpacksDesktopfiles|Original specification]]
 * [[LangpacksDesktopfilesKDE|Original specification for KDE]]

==== Schema files ====

==== Policy files ====

==== Interactive upgrade hooks ====

After certain upgrades (most notably upgrades between Ubuntu releases), there are follow-up actions which should be taken by the user. These are not logically part of the upgrade, but should be dealt with at some point after the upgrade is complete.

They take the form of helper scripts that can be called by the package's `postinst` script, and can be internationalized through language packs.

Here's an example showing how these can be internationalized:

<<Include(InteractiveUpgradeHooks, , from="Starti18nExample", to="Endi18nExample")>>

=== Verifying translation uploads ===

This information is intended to help package maintainers to optimize their packages, so that translation related files get automatically imported into Launchpad.

==== Testing the package locally ====

To test if your package conforms to the requirements of the import script in Launchpad, please do the following:
 1. [[apt://pkgbinarymangler|Install the pkgbinarymangler package]]
 1. In `/etc/pkgbinarymangler/striptranslations.conf` set `enable:` to `true`
 1. Optionally, install any build dependencies: `sudo apt-get build-dep $PKGNAME`
 1. Build the package with `debuild -uc -us -b`
 1. You will find a `<pkgname>-translations.tar.gz` tarball along with the .deb package(s)
 1. Untar that tarball into a temporary directory
 1. Compare if the directory structure and the .pot and .po files conform to the requirements listed below

==== Guidelines ====

If the application uses gettext, like most GTK and GNU applications do:

  * '''Template name and domain''': the filename of the [[#TranslationTemplate|template (.pot file)]] should be the same as what the expected [[#TranslationDomain|translation domain]] is.
  * '''Translation filenames''': the actual [[#TranslationFile|translations (.po files)]] must have a filename of the format `$LANGUAGECODE.po` or `$LOCALECODE.po` (e.g. `de.po`, `pt_BR.po`, `zh_TW.po`)
  * '''Template and translations location''': all translation-related files (the .pot template and its corresponding .po templates) must end up in the same (sub-)directory if possible.
  * '''Multiple domains:''' if the application has multiple translation domains (i.e. multiple .pot files), each .pot file and their corresponding .po files should end up in a separate subdirectory (e.g. po/nautilus/nautilus.pot and po/libnautilus/libnautilus.pot)
  * '''Recommended translations layout''': it is recommended that templates and translations for the main application or library are located in the `po/` directory.
  * '''Recommended help layout''': it is recommended that templates and translations for help files are located in the `help/po/` directory. Note: they will be ignored by Launchpad, since we currently have no infrastructure to deal with them properly.
  * '''Recommended documentation layout''': it is recommended that templates and translations for documentation files are located in the `doc/po/` directory. Note: they will also be ignored by Launchpad for the same reason as the help translations.
  * '''Obsolete translations''': if there are .po files present in the package but no associated .pot template, please remove them, since they cannot be used without a .pot template. There might be a bug in the package or in the source code whereby the .pot file doesn't get generated, so make sure to double-check that as well.
  * '''Valid POT template''': please check if the .pot file has any meaningful content. Empty .pot files should be removed and the package build rules be fixed.
  * '''Templates generated by patches''': if your package contains patches and those patches result into extra templates, like `patches.pot`, please merge those changes into the main template, as we have no use for patch .pot files. Such .pot files should not exist in the final tarball.
  * '''Multiple templates merged into one''': if the package generates multiple .pot files and later merges them into one, only the final .pot file should be present in the final tarball.
  * '''Architecture-dependent templates''': if the package creates the same .pot file in architecture-dependent subdirectories, please consider to merge them, as we can only deal with one single .pot per translation domain.
  * '''Test templates''': test routines and test files should not generate any .pot template or ship any .po file. If such files are present, please remove them.
  * '''Use UTF-8 encoding''': if you have multiple .po files for a language, each in a different encoding, please consider '''renaming''' the ones encoded in UTF-8 to be the main .po file for that language and '''discard''' the other ones. They might be interesting for Windows users, but not for Linux ones, since translations get converted with `iconv` into the encoding the user is runnning anyway. And nowadays UTF-8 is the default encoding on Linux. E.g.: if you have `zh_TW.po` (Big5 encoding), `zh_TW.UTF-8.po` and `zh_TW.cp960.po`, please remove `zh.po` and `zh.cp960.po` and rename `zh_TW.UTF-8.po` to `zh.po`.
  * '''Obsolete `no` locale''': the `no` locale is obsolete. If you find no.po files, please look into it and find out if it's Norwegian Bokmål or Nynorsk. Usually it's Bokmål. Please rename the file to the proper language code: `nb.po` for Bokmål, `nn.po` for Nynorsk. If `nb.po` and `nn.po` are already present, remove `no.po`.
  * '''English US translations''': If you have an `en_US.po` file, please investigate if it really contains translations (i.e. if the source code uses en_GB or another language) and if not, you can safely discard it.

=== Documentation ===

TODO:

 * Converting xml to PO with `xml2po`
 * Launchpad import

=== Packages not using the Gettext format ===

==== Mozilla ====

==== OpenOffice.org ====

== Coding ==

=== Individual programming languages ===

=== Launchpad integration library ===

Ubuntu applications use the [[https://launchpad.net/launchpad-integration|Launchpad integration library]] to provide shortcuts to Launchpad to obtain help, report bugs and submit translations. You can see it in action in any Ubuntu application's Help menu:

{{attachment:LPI.png}}

Here's an example in C of how support for this library was added to Empathy:

{{{
=== modified file 'configure.ac'
--- configure.ac 2009-10-13 03:20:58 +0000
+++ configure.ac 2009-10-13 03:26:58 +0000
@@ -131,6 +131,7 @@
    gstreamer-0.10
    unique-1.0
    gnome-keyring-1 >= $KEYRING_REQUIRED
+ launchpad-integration
 ])
 
 PKG_CHECK_MODULES(LIBNOTIFY, libnotify >= $LIBNOTIFY_REQUIRED)

=== modified file 'src/empathy-main-window.c'
--- src/empathy-main-window.c 2009-10-13 03:20:58 +0000
+++ src/empathy-main-window.c 2009-10-13 18:15:41 +0000
@@ -27,6 +27,9 @@
 #include <gtk/gtk.h>
 #include <glib/gi18n.h>
 
+/* Add launchpad hooks */
+#include <launchpad-integration.h>
+
 #include <libempathy/empathy-contact.h>
 #include <libempathy/empathy-utils.h>
 #include <libempathy/empathy-account-manager.h>
@@ -1424,6 +1427,9 @@
 
  main_window_update_status (window, window->account_manager);
 
+ /* Add launchpad hooks */
+ launchpad_integration_add_ui (window->ui_manager, "/menubar/help/LaunchpadItems");
+
  return window->window;
 }
 

=== modified file 'src/empathy-main-window.ui'
--- src/empathy-main-window.ui 2009-10-13 03:19:42 +0000
+++ src/empathy-main-window.ui 2009-10-13 18:17:20 +0000
@@ -243,6 +243,7 @@
         <menu action="help">
           <menuitem action="help_contents"/>
           <menuitem action="help_debug"/>
+ <placeholder name="LaunchpadItems"/>
           <menuitem action="help_about"/>
         </menu>
       </menubar>
}}}

=== Additional documentation ===

 * [[http://www.gnu.org/software/gettext/manual/gettext.html|Gettext documentation]]
 * [[http://www.gnu.org/s/libc/manual/html_node/Message-catalogs-with-gettext.html| The GNU libc documentation on Gettext]]
 * [[http://live.gnome.org/TranslationProject/DevGuidelines|GNOME l10n guidelines for developers]] and the [[http://live.gnome.org/TranslationProject|GNOME Translation Project]]
 * [[http://docs.python.org/library/gettext.html|Python gettext module documentation]]
 * [[/Build|Adding internationalization support to the build system]]
Line 249: Line 46:
TODO
 * String freeze
 * Translation
 * Translation deadline
 * Language pack updates
''General overview of the Ubuntu translation process, from the translators' point of view''

 * [[/TranslationProcess|Overview of the Ubuntu translation process]]
Line 256: Line 51:
=== The import queue ===
Line 258: Line 52:
All translation templates (.pot files) and translations (.po) end up in the Launchpad Translations import queue before being imported into Launchpad. ''Information related to Launchpad and internationalization, including best practices on how to handle translations of hosted projects''
 * [[https://help.launchpad.net/Translations/YourProject/BestPractices|Best practices on how to set up a project for translation in Launchpad]]
 * [[/Launchpad|Internationalization considerations and general information on Launchpad]]
 * [[http://blog.launchpad.net/translations/automatic-template-generation | Automatic generation of translation templates from branch]]
Line 260: Line 57:
Some of the Ubuntu import queues: == Recipes ==
Line 262: Line 59:
 * [[https://translations.launchpad.net/ubuntu/hardy/+imports?field.filter_extension=pot&field.filter_status=NEEDS_REVIEW|Hardy]]
 * [[https://translations.launchpad.net/ubuntu/intrepid/+imports?field.filter_extension=pot&field.filter_status=NEEDS_REVIEW|Intrepid]]
 * [[https://translations.launchpad.net/ubuntu/jaunty/+imports?field.filter_extension=pot&field.filter_status=NEEDS_REVIEW|Jaunty]]
 * [[https://translations.launchpad.net/ubuntu/karmic/+imports?field.filter_extension=pot&field.filter_status=NEEDS_REVIEW|Karmic]]
''Recipes on how to perform common tasks related to internationalization''
Line 267: Line 61:
Templates generally get approved automatically, but sometimes there is some manual work involved in approving them. The [[https://launchpad.net/~ubuntu-translation-coordinator|Ubuntu Translations Coordinators Team]] manages the import queue. We've got a [[ubuntu-translation-coordinator@lists.launchpad.net|mailing list]] where issues can be discussed and we welcome the help of any community members experienced with translations.

=== If your project is hosted in Launchpad ===



== General Advice ==
If you want to make sure that your packages translations are set up correctly, try running:

 * For GNOME packages:
{{{
cd po; intltool-update -m}}}
and be sure to either fix the issues or report them upstream.

Note: ''this is only worth trying if the package uses intltool for i18n (i.e. look for intltool.m4 in the top-level source directory)''

 * For GNU packages:

 * For KDE packages:

Sometimes empty `.po` files caused packages to build from source.

== Kubuntu-specific notes ==
 * [[/RecipeVerifyingTranslationUploads|Verifying translation uploads]]
 * [[/RecipeIntltoolCheck|Checking out translations in intltool-based packages]]
 * [[/RecipeAddingLPI|Adding Launchpad integration for bugs, help and translations to an application's help menu]]
Line 293: Line 67:
CatagoryTranslations CategoryTranslations

Ubuntu Internationalization Guide

This document tries to outline:

  • How Ubuntu deals with translations (and i18n in general) from a developer's perspective
  • Which tools are interesting for developers to make sure that translations and i18n work fine

Getting Started

General information

General information on the tools and processes related to the internationalization of Ubuntu

Coding

Information on writing or modifying internationalized applications for Ubuntu

Packaging

Information on packaging internationalized packages for Ubuntu

Documentation

Information on how to adapt the build system to include internationalization support

Build system

Information on how to adapt the build system to include internationalization support

Translation process

General overview of the Ubuntu translation process, from the translators' point of view

Launchpad

Information related to Launchpad and internationalization, including best practices on how to handle translations of hosted projects

Recipes

Recipes on how to perform common tasks related to internationalization


CategoryUbuntuDevelopment CategoryTranslations

UbuntuDevelopment/Internationalisation (last edited 2010-08-18 14:21:17 by 140)