Ubuntu Wiki

Ubuntu Open Week - Translations and Internationalization with Launchpad - MikeRooney - Fri, Nov 7th, 2008

(01:01:18 PM) mrooney: I am Michael Rooney, and I'll be talking about setting up a project for translations, getting translations, and then integrating them into your project, with the help of Launchpad.
(01:01:19 PM) andi is now known as Guest44770
(01:01:31 PM) mrooney: I won't be talking explicitly about the overall Ubuntu translation effort, instead more on a per-project level.
(01:01:43 PM) mrooney: Including what developers can do, how users can help, and how multi-lingual speakers can help. Really anyone can contribute to the process.
(01:02:01 PM) mrooney: But I will try to address any questions to the best of my ability, don't forget to ask in #ubuntu-classroom-chat, prefacing them with QUESTION and/or mrooney :)
(01:02:16 PM) mrooney: I have also recently overviewed this process on my blog at http://mrooney.blogspot.com/ , so if you want to ask questions later feel free to head over there and leave a comment.
(01:02:30 PM) mrooney: Okay so first, let's give a brief overview, what is internationalization (i18n) and localization (l10n)?
(01:02:37 PM) mrooney: Who here already has an idea?
(01:03:18 PM) mrooney: Okay well good, I shall explain!
(01:03:21 PM) mrooney: Firstly, they are often abbreviated as above because of the number of letters between the first and last letters.
(01:03:41 PM) mrooney: i18n is the process of making locale-specific elements of your application translatable.
(01:04:02 PM) mrooney: l10n is then integrating translations into the project so that users of those languages see their language by default instead of English.
(01:04:19 PM) mrooney: The scope of this is generally more than just language, and includes the way currencies are displayed, among other things. Everything to do with another culture.
(01:04:27 PM) mrooney: For a more thorough explanation, see http://en.wikipedia.org/wiki/Internationalization_and_localization
(01:04:33 PM) mrooney: I am now going to cover why and how to do this, but first let me ask if there are any questions before going on, such as about the goals of this process?
(01:05:26 PM) mrooney: Basically, we want people speaking other languages to be able to use applications!
(01:05:44 PM) mrooney: If you don't localize, people not speaking English can't use your application!
(01:05:47 PM) mrooney: And that's no fun.
(01:06:04 PM) mrooney: Plus to be included into Ubuntu, at least the main repository, a project will need to have translations.
(01:06:31 PM) mrooney: So now the first step is i18n, making a project translatable!
(01:06:48 PM) mrooney: I am going to start out with a really simple python example, so anyone can follow allow if they want.
(01:06:54 PM) mrooney: *along
(01:07:08 PM) mrooney: This is where developers of projects can help, but it is also simple enough that a casual user of the application could contribute to this process as well.
(01:07:26 PM) mrooney: Basically, we need to wrap all user-visible text under a translation lookup layer.
(01:07:39 PM) mrooney: Initially this will be slightly technical but I think we can handle it :)
(01:07:46 PM) mrooney: We will use gettext (http://en.wikipedia.org/wiki/Gettext) for this.
(01:08:12 PM) mrooney: Let's use a simple python program. We will create a file called "example.py"
(01:08:34 PM) mrooney: Now we will put one line in it: print "Hello, world!"
(01:08:44 PM) mrooney: When run, this will print "Hello, world!" to the console!
(01:08:53 PM) mrooney: But there is a problem, does anyone see it?
(01:09:34 PM) mrooney: Okay the problem is, anyone not speaking english won't understand this program!
(01:10:08 PM) mrooney: (while the examples I use are Python, any modern language should have very similar equivalents)
(01:10:23 PM) mrooney: First let's make python aware of what language the user is using!
(01:10:33 PM) mrooney: Let's add one line to the top
(01:10:47 PM) mrooney: import locale; locale.setlocale(locale.LC_ALL, "")
(01:10:58 PM) mrooney: What we do here is import the locale module and tell it, for all aspects of a locale: language, currency, dates, etc. (LC_ALL), use the users default language ("")
(01:11:23 PM) mrooney: Any questions there?
(01:11:51 PM) mrooney: Now we need to add the translation layer for text:
(01:11:56 PM) mrooney: import gettext; gettext.install("yourAppName", "locales")
(01:12:12 PM) mrooney: This tells gettext to install itself and use "yourAppName" as the scope
(01:12:29 PM) mrooney: the name you use here won't matter unless you get into more complex translations
(01:12:43 PM) mrooney: the "locales" is telling gettext where to attempt to find translations
(01:12:57 PM) mrooney: ie a folder named "locales" wherever test.py is located.
(01:14:25 PM) mrooney: QUESTION: Is the «import locale; locale.setlocale(locale.LC_ALL, "")» chunk really necessary? I've a Python project using gettext but I don't do that there; could this cause any problem?
(01:15:00 PM) mrooney: It is definitely useful for other aspects of l10n including currency and dates, though gettext may be able to pick up on the default language by itself
(01:15:48 PM) mrooney: If you are only localizing your application with translations and you have found that it works without that, it is probably fine!
(01:16:15 PM) mrooney: Okay, now, what gettext.install has done is create a function, named _ (the underscore key), so that it is quick and easy to type and spot.
(01:16:29 PM) mrooney: So now we can change: print "Hello, world!" to print _("Hello, world!")
(01:16:43 PM) mrooney: What this does is give "Hello, world!" to gettext and ask, do you have a translation for this text, for the users language?
(01:17:03 PM) mrooney: Now in the case of English, it will just give back the same text.
(01:17:21 PM) mrooney: So if you are following along you can run it now, and it will still print "Hello, world!"
(01:17:42 PM) mrooney: So that is the first step, wrapping all user-visible strings in _(), which an intermediate user could probably take a stab at, by downloading a projects source and searching for text they find in the application, assuming it isn't already internationalized, of course.
(01:18:23 PM) mrooney: Okay, now we are through the technical part, does that process make sense? Any questions?
(01:18:55 PM) mrooney: QUESTION: C++ equivalent?
(01:19:12 PM) mrooney: The wrapping of the strings is going to be the same in almost any language
(01:19:57 PM) mrooney: The only different part is going to be the initializing line, I don't know it off hand for other languages but it should be easy enough to find online
(01:20:08 PM) mrooney: Sorry I can't be more specific, there
(01:20:26 PM) mrooney: Okay, now we need to make a translation template that translators can use!
(01:20:48 PM) mrooney: This is fairly easy and involves running xgettext on the files you updated.
(01:21:10 PM) mrooney: There are language specific programs, python has one, but I am using xgettext as it should work with Python, Java, C++, and others
(01:21:36 PM) mrooney: so in this case, you'll want to run "xgettext example.py > messages.pot"
(01:22:02 PM) mrooney: If you have a bunch of files you can string them, such as "xgettext file1.py file2.cpp > messages.pot"
(01:22:14 PM) mrooney: This will create a template called messages.pot, which contains all the strings necessary for translation.
(01:22:41 PM) mrooney: Now from this, an experienced translator could make a translation for you.
(01:23:04 PM) mrooney: But to make it as easy as possible, we want to use Launchpad's Rosetta service, which provides a simple web UI to do this!
(01:23:22 PM) mrooney: If the project isn't already in Launchpad, go to launchpad.net to register, which is completely free.
(01:23:31 PM) mrooney: From here on out I'll be using my project, wxBanker, as an example
(01:23:41 PM) Intey: link plz
(01:23:52 PM) mrooney: Edit your projects details (in my case: https://launchpad.net/wxbanker/+edit) and check the box that says "Translations for this project are done in Launchpad", scroll to the bottom, and click "Change".
(01:24:04 PM) mrooney: You probably won't be able to edit my project :)
(01:24:18 PM) mrooney: but the general page is https://launchpad.net/wxbanker
(01:24:39 PM) mrooney: Now click the Translations tab of your project in Launchpad and upload the .pot file you just generated with xgettext.
(01:24:53 PM) mrooney: If this is your first time doing so, it will need to be reviewed by a human, and will take perhaps a day to get approved, at which point you will receive an email letting you know it has been and you are good to go.
(01:25:09 PM) mrooney: Once it has been approved, your project can now be translated!
(01:25:25 PM) mrooney: So go to say https://translations.launchpad.net/wxbanker, and if your language settings are configured for more than English, you can start translating!
(01:25:51 PM) mrooney: (there is "Select languages" option in the lower right)
(01:26:10 PM) mrooney: QUESTION: Could you explain why new translation templates have to be manually reviewed before being accepted, please?
(01:26:26 PM) mrooney: So you have upload messages.pot, which is basically a text file containing all the strings
(01:26:56 PM) mrooney: However before it goes into Launchpad, someone needs to make sure you actually generated it correctly, that it looks sane, et cetera
(01:27:29 PM) mrooney: So that someone doesn't say try to make their own by hand or otherwise do something silly
(01:27:56 PM) mrooney: Perhaps someone wasn't wrapping their strings correctly, or another issue. Maybe the original strings aren't in English!
(01:28:05 PM) mrooney: So in this way a human needs to verify it is sound.
(01:28:30 PM) mrooney: Make sense?
(01:28:39 PM) Intey: does.
(01:28:53 PM) RainCT: Yep, thanks :)
(01:28:55 PM) mrooney: So now, how can you get it translated?
(01:29:16 PM) mrooney: Well, if you know another language you can tell launchpad via the language settings button
(01:29:35 PM) mrooney: Then you translate into that language yourself.
(01:29:46 PM) mrooney: However, this probably isn't the case for a lot of people, and one language won't be enough.
(01:30:09 PM) mrooney: A very interesting feature of Launchpad however is that it will show translations from other projects, so you can do common ones yourself without knowing the language.
(01:30:20 PM) mrooney: For example if "Hello" is in your template, it will tell you that other projects use "Hola" for Spanish and allow you to select that.
(01:30:46 PM) mrooney: So in this way you can make all the basic File, Help, Save, and such translated
(01:31:38 PM) mrooney: There are a lot of projects in Launchpad, you might be surprised to find that translations already exist for a lot of strings!
(01:31:48 PM) mrooney: QUESTION: will there be a special howto available for those of us who dont write code at all, consider themselves power users though and are literate enuff...in short: ppl that just want to translate, efficient, sound, clean. supportive....blah..?
(01:32:09 PM) mrooney: Intey: do you mean people that know another language and want to translate from English into that?
(01:32:26 PM) Intey: exactly^
(01:33:07 PM) mrooney: In that case what you will want to do is probably is join ubuntu-translators (https://translations.launchpad.net/+groups/ubuntu-translators)
(01:33:32 PM) mrooney: However you will probably need to demonstrate you have translation experience as those users can translate all of Ubuntu (and we need definite help there!)
(01:33:48 PM) mrooney: So you may want to find projects (like wxBanker) which is open and allows anyone to translate
(01:34:11 PM) mrooney: So I would start off volunteering to translate new projects
(01:34:43 PM) mrooney: There are going to be a lot of new UIs made for Jaunty, for example, I imagine, so if you want to start asking around how you can help translate those
(01:34:50 PM) mrooney: QUESTION: What do we have to do in case of changes made to the source code? Should we just freeze the project and wait for the translatians to be done and then release the project?
(01:35:05 PM) mrooney: Well, the advantage of gettext is that it won't cause any regressions
(01:35:22 PM) mrooney: So you can wrap all your strings and generate a template , and it will still function the same
(01:35:49 PM) mrooney: So you don't have to freeze the project development, you can release it internationalized with no translations for example
(01:35:55 PM) mrooney: Does that make sense?
(01:36:22 PM) weboide: yes, and what happens when you upload new/updated *.pot files to LP?
(01:36:44 PM) mrooney: weboide: so when you upload a new template, any strings already translated are naturally not lost
(01:37:09 PM) mrooney: so if you forgot to wrap say the "Help" menu (a real example that happened to me :)
(01:37:22 PM) mrooney: you can wrap that in the code, generate a new template, upload it, and now there is one new string
(01:37:55 PM) mrooney: When people translate that string (or you use suggestions), just download the translations again and replace the old ones
(01:38:15 PM) mrooney: and now you have that translation for the new string
(01:38:19 PM) weboide: Thanks :)
(01:38:48 PM) mrooney: QUESTION: How are translations done for static content like SVG images, themes and books? Can those be helped via Launchpad as well?
(01:38:55 PM) mrooney: It depends :)
(01:39:09 PM) mrooney: As long as you can generate a list of text that needs to be translated, yes
(01:39:48 PM) mrooney: Picklesworth: It should also be noted, on the topic of translations, that GTK offers a lot of Stock content for things like menu items and buttons, which magically translate themselves too
(01:39:55 PM) mrooney: Yes, that is a good point
(01:40:06 PM) mrooney: And addresses what RainCT asked earlier I believe, with setting up the locale module
(01:40:46 PM) mrooney: When you do that, stock names including File, Edit, About items will often be automatically translated if the framework suppots it
(01:41:38 PM) mrooney: QUESTION: how do you handle constraints (like: there's only so much room for a line of text)
(01:41:54 PM) mrooney: This gets more into UI design, but it is a great point
(01:42:18 PM) mrooney: If you are using a UI framework properly, you ideally won't have such constraints
(01:42:37 PM) mrooney: It is important to almost never use absolute positioning, for example
(01:42:45 PM) mrooney: this will cause painful issues
(01:43:15 PM) mrooney: You want to design it from the ground up ideally, to make minimal assumptions about length, such as using controls that support word-wrapping
(01:44:18 PM) mrooney: GTK and wxWidgets for example have the concept of sizers
(01:44:37 PM) mrooney: And you will want to use layout elements like that
(01:44:44 PM) mrooney: samgee: does that answer your question at all?
(01:44:58 PM) samgee: I guess so
(01:45:40 PM) Intey: wait, as a end-translator, does launchpad offer ways to see the maximum character lenght or somesuch? if not..blueprint?
(01:45:46 PM) mrooney: Basically you want to attempt to eliminate constraints by word-wrapping and laying out without making assumptions
(01:46:14 PM) mrooney: Intey: You can add comments to each string, which I didn't mention
(01:46:33 PM) mrooney: Intey: but for example you might put // TRANSLATORS: this string can only be 100 chars
(01:46:37 PM) mrooney: before the string
(01:46:53 PM) mrooney: and if you pass something like --coments=// to xgettext, it will add that to the template and show up on Rosetta
(01:47:06 PM) mrooney: I don't remember the exact xgettext argument, it probably isn't exactly that
(01:47:14 PM) mrooney: But in that way you can inform translators of specific knowledge
(01:47:22 PM) mrooney: and context
(01:48:04 PM) mrooney: So that can help address samgee's issue perhaps as well. When you have to have a constraint, you can at least make translators aware
(01:48:33 PM) mrooney: Let's see before I explain getting the translations into your project, which is easy, there was one more ?
(01:48:43 PM) mrooney: Intey: I sometimes find myself accidentoly adding translations that have already been suggested, whilst proof-reading them...
(01:48:50 PM) mrooney: Intey: as a result fellow translaters think Im stealing their work ...sort of
(01:49:08 PM) mrooney: Rosetta will inform you of whether you can freely use a suggested translation or not
(01:49:41 PM) mrooney: It will put a yellow warning sign next to a suggestion, if Launchpad isn't sure that you can freely use it
(01:49:53 PM) mrooney: and in that case you would need to ask or review the license
(01:50:33 PM) mrooney: And conversely when you make translations in an open-source project, you are typically allowing OTHER translators to use what you used
(01:50:36 PM) mrooney: in another project
(01:50:56 PM) mrooney: Okay lets go to the final step!
(01:51:00 PM) mrooney: Once you have some translations in Launchpad, you are basically done and it is super easy to integrate them into the project!
(01:51:25 PM) mrooney: Visit the translations page for your project in Launchpad, and click the "Download translations" link on the right.
(01:51:39 PM) mrooney: From here ensure "Everything" is selected, and change the Format to "MO format" (what gettext will use). Now click "Request download".
(01:51:48 PM) mrooney: This isn't always instant, although you will be emailed with a link to the file if it isn't.
(01:51:55 PM) mrooney: Now, remember that "locales" folder we told gettext about earlier?
(01:52:10 PM) mrooney: Once you downloaded your file "launchpad-export.tar.gz", extract that into where your source is, and rename it to "locales".
(01:52:20 PM) mrooney: Bam! Now gettext will find your translations and use them instead of English when available
(01:52:26 PM) mrooney: Bam! Now gettext will find your translations and use them instead of English.
(01:52:43 PM) mrooney: whoops, well twice for emphasis
(01:52:50 PM) mrooney: Now anyone using a language that you have a translation for, will now see the translations instead of English!
(01:52:57 PM) mrooney: So that is the process from start to finish
(01:53:30 PM) mrooney: Any other questions?
(01:53:56 PM) mfoniso: yeah...
(01:54:08 PM) mrooney: RainCT: Did I address your locale.setlocale question, on the advantages of how frameworks such as GTK and wxPython will pick up on that and translate stock strings automatically?
(01:54:21 PM) mrooney: QUESTION: what are the limitations of gettext? Are there any alternatives that provide advantages (I remember a Mozilla talk at FOSDEM about something better in the works)
(01:54:48 PM) mrooney: One limitation is that strings are translated in an all or nothing approach
(01:55:02 PM) mrooney: either you have a translation that exactly matches or it doesn't get translated
(01:55:06 PM) RainCT: mrooney: Yes, thanks :). (It isn't an issue for my project as it uses pygame, but that's good to know).
(01:55:33 PM) mrooney: so if you have a translation for "Hello" and "world", you still need one for "Hello world"
(01:55:37 PM) mrooney: if you use all of those
(01:55:41 PM) mrooney: mfoniso: a question?
(01:56:03 PM) mrooney: QUESTION: is automatic translation an option (babelfish etc.) an option if there are not enough volunteers?
(01:56:16 PM) mrooney: not that I know of, and that would generally produce poor translations
(01:56:34 PM) mrooney: however, using the suggestions from Rosetta is almost as good and will get you better results
(01:56:39 PM) mfoniso: mrooney: I've asked in #ubuntu-classroom-chat
(01:56:44 PM) mrooney: however it is a one-at-a time operation
(01:57:19 PM) mrooney: QUESTION:how does one go about doing translations into other langauges, for ubuntu
(01:57:27 PM) mrooney: Okay let's see if I can address this as my last question
(01:57:50 PM) mrooney: If you look at https://translations.launchpad.net/ubuntu, you will see we need a LOT of translations!
(01:58:01 PM) mrooney: So help in this area is hugely appreciated and needed
(01:58:10 PM) mrooney: and rewarding because you can see your own translations :)
(01:58:32 PM) mrooney: to be able to translate here, you need to be a member of ubuntu-translators
(01:58:56 PM) mrooney: So you'll want to look into joining https://translations.launchpad.net/+groups/ubuntu-translators
(01:59:07 PM) mrooney: which will allow you translate almost all of Ubuntu that supports it
(01:59:41 PM) mrooney: So find the sub team that you want to translate in
(01:59:44 PM) mfoniso: nice
(02:00:02 PM) mrooney: And you can ask to join their group
(02:00:15 PM) mrooney: Okay, I hope that wraps it up!

CategoryTranslations