== App Developer Week -- From English to any language: internationalizing your apps -- dpm -- Mon, Apr 11th, 2011 == {{{#!irc [19:00] hey, hello everyone! [19:01] thanks for joining in this session on how to internationalize your applications === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu App Developer Week - Current Session: From English to any language: internationalizing your apps - Instructors: dpm [19:02] Logs for this session will be available at http://irclogs.ubuntu.com/2011/04/11/%23ubuntu-classroom.html following the conclusion of the session. [19:02] ok, now that classbot is done... [19:03] first of all, thanks to tomeu for a great session [19:03] And now let's get started with translations [19:03] First the introductions [19:03] I'm David Planella, and I work in the Community team at Canonical as the Ubuntu Translations Coordinator [19:04] Usually I work more on the community side of things, with the always awesome Ubuntu translation teams [19:05] But today I've put my developer hat to show you how easy it is to get your app ready to speak a multitude of languages [19:05] and set up so that the community can translate it. [19:05] Regardless of the programming language, the process of adding internationalization support to an application is not only fairly easy [19:06] but also, on a high level view, the same for all programming languages. [19:06] This means that after this session you should have a pretty good overview on what it takes to make your application translatable [19:06] and you can apply this to any programming language, slightly adapting the syntax, of course. [19:07] In order for you to see how it all fits together, I've based the talk on a common framework that can get you quickstarted in just a few minutes [19:07] I've used the Python programming language and Quickly [19:08] https://wiki.ubuntu.com/Quickly [19:08] Let's start with some background concepts to make it easier to understand the steps we'll be doing later on [19:09] So let's have a quick look at the main players involved in the internationalization game: [19:09] [19:09] Background Concepts [19:09] =================== [19:09] [19:09] GNU Gettext [19:09] ----------- [19:09] Gettext is the underlying and most widely used technology to enable translations of Open Source projects. [19:09] It defines a standard format of translation files translators can do their work with (PO files, more on them in a minute) [19:10] and lets applications load those translations compiled in a binary format (MO files) at runtime. [19:10] It has implementations for many programming languages, and amongst them, of course, Python. [19:10] You'll find that the comprehensive gettext manual at http://www.gnu.org/software/gettext/manual/gettext.html can be a very useful reference, [19:10] The Python implementation of the gettext API is what we'll use to internationalize our project with Quickly today. [19:11] Needless to say, it also comes with some nifty documentation at http://docs.python.org/library/gettext.html [19:11] * {i} In short, gettext does all the heavy lifting involved in exposing your application for translation and loading the translations for the end user [19:12] [19:12] intltool [19:12] -------- [19:12] Intltool is a higher level tool that adds functionality to gettext by allowing the extraction of translatable strings from a variety of file formats [19:13] It has also become a standard tool when implementing internationalization for OSS projects. [19:13] Nearly all (if not all) GNOME projects, for example, use intltool. [19:14] * {i} intltool handles the translations of things such as the desktop shortcut of your application [19:14] [19:14] python-distutils-extra [19:14] ---------------------- [19:14] Python-distutils-extra is a python package that makes it easy to integrate themable icons, documentation and gettext based translations in your python install and build tools, and it's basically an enhancement to python-distutils. [19:15] The project's page is at http://www.glatzor.de/projects/python-distutils-extra/ [19:15] * /!\ Note that this tool is Python-specific. I'm mentioning it here because we're going to be talking of a practical example with Python. If your application were a C application you'd probably use autotool rules to achieve the same result [19:15] [19:15] The three above technologies (gettext, intltool, python-distutils-extra) are transparently used by Quickly, so we won't get into much more detail for now. [19:16] I just want you to get an idea of what we're talking about [19:16] There are also more aspects involved in internationalizing applications, such as font rendering, input methods, etc., but this should get you started for now. [19:16] [19:16] Quickly [19:16] ------- [19:16] I'll be very brief here and let you figure out more on quickly as we go along [19:16] For now, it will suffice give you a teaser and tell you that it is the tool which brings back the fun in writing applications! ;-) [19:17] [19:17] Finally, a tool that is not strictly needed for internationalization (or the shorter form: i18n), but that can help you build an active translation community around your project [19:18] which will be the next step after your project adds i18n support [19:18] [19:18] Launchpad Translations [19:18] ---------------------- [19:18] Launchpad Translations (https://translations.launchpad.net/) is the collaborative online tool which allows translation communities to be brought together and translate applications online through its web UI. [19:18] Apart from the very polished UI to provide translations, it has other nice features such as message sharing across project series (translate one message in a series and it instantly propagates to all other shared series), [19:19] global suggestions (suggestions of translations across _all_ projects in Launchpad), automatic imports of translations and automatic commits to bzr branches, several levels of permissions, and a huge translator base. [19:20] On the right hand side of the URL I gave you you can see that there are quite a lot of projects using Launchpad to make translations easy both for developers and translators. [19:20] Bear with me: we're nearly there - let's also quickly trow in and review a couple of concepts related to the gettext technology [19:20] [19:20] Gettext: MO files [19:20] ----------------- [19:21] The message catalog, or MO file (for Machine Object) is the binary file that is actually used to load translations in a running system. [19:21] It is created from a textual PO (more on that in a bit), which is used as the source, generally by a tool called msgfmt. [19:21] Message catalogs are used for performance reasons, as they are implemented as a binary hash table that is much more efficient to look up at runtime than textual PO files [19:22] * {i} .mo files are the binary files installed in the system where the application loads the translations from, using gettext [19:22] [19:22] Gettext: PO files [19:22] ----------------- [19:22] PO file stands for Portable Object file, and are the textual files translators work with to provide translations. They are plain text files with a special format: [19:22] msgid "English message" [19:22] msgstr "Traducció al català " <- Translated string [19:22] (message pairs containing the original messages from the application, and its corresponding translation) [19:23] You can see an example of a PO file here: [19:23] http://bazaar.launchpad.net/~synaptic-developers/synaptic/trunk/view/head:/po/zh_CN.po [19:23] * {i} Translators provide translations in PO files. If they use an online translation system they won't directly work with them, but your project sources will still contain them [19:23] * {i} In each application source tree there is generally a PO file per language, named after the language code. E.g. ca.po, de.po, zh_CN.po, etc. [19:23] [19:23] Gettext: POT files [19:23] ------------------ [19:23] Once your project has added i18n support, you'll need to give translators an updated list of translatable messages they can start translating. [19:23] You'll also need to update this list whenever you add new messages to your application. [19:24] You achieve that through POT files, or simply templates in l10n (localization) slang (Portable Object Template). [19:24] They are textual files with the same format as PO files, [19:24] but they are empty of translations and are used as a template or stencil to create PO files from. [19:24] * {i} There are special tools to update templates. Generally intltool is used, often called from a build system rule or a higher level tool such as python-distutils-extra [19:24] [19:24] Gettext: Translation domain [19:24] --------------------------- [19:25] The translation domain is a unique string identifier assigned by the programmer in the code (usually in the build system) [19:25] and used by the gettext functions to locate the message catalog where translations will be loaded from. [19:25] The general form to compute the catalog’s location is: [19:26] locale_dir/locale_code/LC_category/domain_name.mo [19:26] which in Ubuntu expand generally to /usr/share/locale/locale_code/LC_MESSAGES/domain_name.mo [19:26] The locale_code part refers to the language code for the particular language. As an example, when using Nautilus in a Catalan locale with Ubuntu, [19:26] the gettext functions will look for the message catalogue at: [19:26] /usr/share/locale-langpack/ca/LC_MESSAGES/nautilus.mo [19:27] That's where the translations for your application will be installed and searched for [19:27] Note that for your app this location might be slightly different: [19:27] /usr/share/locale/ca/LC_MESSAGES/myapp.mo [19:28] * {i}The corresponding translation template should have the same translation domain in its filename, e.g. nautilus.pot. [19:28] * {i} The translation domain must be unique across all applications and packages. I.e. something generic like messages.pot won’t work. [19:28] [19:29] Ok, done with the concepts, let's get down to work and to questions [19:29] Generic Steps to Internationalize an Application [19:29] ================================================ [19:29] * Integrate gettext into the application. Initialize gettext in your main function, most especially the translation domain [19:30] * Integrate gettext into the build system. There are generally gettext rules in the most common build systems. Use them. [19:30] * Mark translatable messages. Use the _() gettext call to mark all translatable messages in your application [19:31] * Care for your translation community. Not necessarily a step related to adding i18n support, but you'll want an active and healthy translation community around your project. Keep the templates with translatable messages up to date. Announce these updates and give translators time to do their work before a release. Be responsive to feedback. [19:31] [19:31] Hands-on: creating an internationalized app [19:31] =========================================== [19:31] Ok, enough theory, let's have a go at using quickly to create your first internationalized application [19:32] You can install Quickly on any recent Ubuntu release by simply firing up a terminal and executing: [19:32] sudo apt-get install quickly [19:33] Once you've done that, you can run quickly to create your first project: [19:34] quickly create ubuntu-application awesometranslations [19:34] (if you like, substitute 'awesometranslations' by your favourite project name) [19:34] We've just told Quickly to use the ubuntu-application template, and to call what is created "awesometranslations" [19:35] you should probably have an open dialog from your new app in front of you. [19:35] Quickly has created all that for you! [19:35] You can close the dialog to continue [19:35] What Quickly did was to copy over basically a sample application, and do some text switcheroos to customize the app [19:36] What you could see there was the ui containing some text that needs translation. [19:36] To start making change to your app, go to the directory where it's stored. Generally by simply running [19:36] cd awesometranslation [19:37] You can then edit your code with $ quickly edit, change the UI with $ quickly glade, and try your changes with $ quickly run [19:37] You can save your change with $ quickly save [19:37] Finally, to package, share, release your apps so that other will be, with the following commands (not all are necessary): $ quickly package / $ quickly share / $ quickly release [19:37] As it stands now, the application has nearly all you need to make it translatable [19:38] which is the great thing about quickly [19:38] From now on, while I'll let you play and investigate the application you've created, we'll be looking at the one I created for the purpose of this session [19:39] and I'll show you the i18n bits [19:39] so that you can add them to your existing applications if you want [19:39] For new applications, I'd simply recommend you to use quickly [19:39] and start from there [19:40] which will set up everything for you, so that you can forget about it and concentrate on all those new cool functions your new app is going to provide :-) [19:40] So let's have a look at: [19:40] http://bazaar.launchpad.net/~dpm/+junk/awesometranslations/files [19:41] Notice that we could just call the session finished at this point, as quickly did all the job for us :) [19:41] Also notice how easy it is. I've just created and pushed the application to Launchpad a few minutes ago [19:42] Remember we were talking about PO and POT files? [19:42] They are right there, under the po/ folder: [19:42] http://bazaar.launchpad.net/~dpm/+junk/awesometranslations/files/head:/po/ [19:43] the po/ folder could be called something else, but it is customary to call it like that, as some tools rely on this convention [19:43] Notice the .pot file called the same name as your app [19:43] and an example translation file (ca.po) submitted by a translator [19:44] Now to the interesting bits: [19:44] Remember the generic steps for internationalization we were talking about earlier on? [19:44] * Initializing gettext: [19:44] http://bazaar.launchpad.net/~dpm/+junk/awesometranslations/view/head:/awesometranslations/__init__.py#L8 [19:45] so here we include the gettext module [19:45] and we define a function called simply _() [19:45] And finally we define the translation domain [19:46] which will be the name of the .mo file installed in the system and the name of the .pot file [19:47] * Integrating gettext in the build system: [19:47] http://bazaar.launchpad.net/~dpm/+junk/awesometranslations/view/head:/setup.py#L13 [19:47] Here the integration happens automagically by using python-distutils-extra [19:47] C programs using autotools might need a more complex integration [19:48] * Mark translatable messages: [19:48] http://bazaar.launchpad.net/~dpm/+junk/awesometranslations/view/head:/awesometranslations/__init__.py#L23 [19:49] For every message you want to expose for translation, you simply have to wrap it in the _() function, e.g. _("Translatable message here") [19:49] And that's basically it, really [19:50] easy, isn't it? [19:50] ok, so we're running out of time, let's see if there are questions! [19:52] There are 10 minutes remaining in the current session. [19:52] bdfhjk asked: What is the best way to translate QT application? Gettext or QTLinquist? [19:52] bdfhjk asked: What is the best way to translate QT application? Gettext or QTLinquist? [19:52] Tough question :) [19:53] Both Gettext and QT Linguist are excellent i18n frameworks [19:53] With similar functionality [19:53] But I would personally use gettext [19:54] Because it is framework-agnostic and used by the vast majority of Open Source projects [19:54] Not only that, but most online translation tools rely on gettext [19:54] KDE itself uses gettext, for example [19:54] bulldog98_konv asked: what’s the difference between GNOMEs and KDEs handling of translations in code? [19:55] Not much really [19:55] As I said, both KDE and GNOME use gettext [19:55] The majority of GNOME is written in C, and KDE in C++ [19:56] I gather that in KDE they wrap Qt Linguist calls through kdelib to actually use gettext to load the actual translations [19:56] bdfhjk asked: Is Gettext working in windows? [19:57] There are 5 minutes remaining in the current session. [19:57] Yes, gettext works in any platform where glibc can run, including Windows [19:58] There is still time to answer one last question if you've got one [19:59] Ok, so I think I'll use the last minutes to thank everyone for their participation, and remind you that if you've got any questions on translations, feel free to ping me any time! [19:59] I usually hang out on #ubuntu-devel }}}