Ubuntu Open Week - Translations with Rosetta - Mon, Nov 27, 2006
see also Wednesday Session
08:04 jordi Ok, so for those who don't know me, I'm Jordi Mallach, and I've been involved with the Rosetta team trying to be the link between the development team and the Ubuntu translators and rosetta users 08:06 jordi let's get moving 08:06 jordi The Rosetta Translation Portal 08:06 jordi Rosetta is one of the components that make up Launchpad, 08:06 jordi Canonical's service platform. 08:06 jordi Launchpad is made up of five major components: a bug tracker, a 08:06 jordi request tracker, a specification tracker, a "source code" 08:06 jordi supermirror and Rosetta, a web-based translation portal. 08:06 jordi Christian Reis will talk tomorrow about Launchpad in general, so 08:06 jordi let's focus on Rosetta. 08:06 jordi Rosetta's aim is to make translation of Free Software as easy and 08:06 jordi non-technical as it can get. The Rosetta team has been working on 08:06 jordi creating an interface which hides the specifics of the Gettext PO 08:06 jordi file format, which is the standard for translating Free Software, 08:06 jordi thus lowering the barrier so anyone with a reasonable knowledge 08:06 jordi of English can help out with the translations of their favourite 08:06 jordi project into their mother tongue. 08:07 jordi (please say if I'm too fast, I'm worried about lack of time) 08:07 jordi Rosetta is the main translation system of Ubuntu Linux, and is 08:07 jordi the source of all translations which appear in the Ubuntu 08:07 jordi releases, and in the frequently updated langpacks. Rosetta is 08:07 jordi also designed to help program authors getting their applications 08:07 jordi translated. 08:07 jordi A close look on the Gettext PO file format 08:07 jordi ========================================== 08:07 jordi Most of the software in your desktops use a standard translation 08:07 jordi interface called GNU gettext, which is in charge of showing the 08:07 jordi applications in the language the user has chosen. Application 08:07 jordi programmers need to take care of marking all the user-visible 08:07 jordi messages (or strings, as the initiated tend to call them) with a 08:07 jordi special marker which can be extracted to plain text ".po" files. 08:07 jordi We translators use these files to translate the applications. 08:08 jordi Let's look at how a PO file looks. I've put some examples in 08:08 jordi http://pusa.informat.uv.es/~jordi/ubuntu-school/ 08:08 jordi Have a look at the ubuntu-school.pot file. A POT file is a "PO 08:08 jordi Template", that is, an empty PO file ready to be translated. 08:08 jordi Looking at the contents of the file, you can see the format is 08:08 jordi pretty straight forward: each original string in English (a 08:08 jordi msgid) has its corresponding translation (msgstr). While simple, 08:08 jordi the po format is quite fragile. One missing quote, and your 08:08 jordi entire application build will fail with a syntax error. There are 08:08 jordi several very popular PO file editors which help the editing 08:08 jordi process: KBabel, PoEdit, GTranslator, Emacs PO-mode... 08:09 jordi Rosetta goes one step further in easing the translation of these 08:09 jordi PO files, using a clean, web-based interface which hides the 08:09 jordi format, presenting only sets of string/translation pairs that you 08:09 jordi can fill up. Once the work is done, it's stored in its database 08:09 jordi where the information can be exported or shared among other 08:09 jordi projects. 08:09 jordi Using Rosetta's Web Interface 08:10 jordi Rosetta is, as hinted before, divided in two main branches: one 08:10 jordi serves to translate the applications of the people who request 08:10 jordi it. For example, the Gobby collaborative editor is being 08:10 jordi translated by Rosetta contributors, after its authors requested 08:10 jordi us to set it up for them in Rosetta. On the other hand, Rosetta 08:10 jordi is the platform from where Ubuntu gets all its translations. 08:10 jordi We'll focus on Ubuntu a bit more now. 08:10 jordi Ubuntu translations revolve around the Ubuntu translation teams, 08:10 jordi which coordinate and produce the translations which get shipped 08:10 jordi with every new version. 08:10 jordi https://launchpad.net/rosetta/groups/ubuntu-translators 08:11 jordi Here you'll see a list of teams which belong to the Ubuntu 08:11 jordi translation teams. While Rosetta is open enough to let everyone 08:11 jordi with a Launchpad account contribute, there is need for some 08:11 jordi access control, to protect quality, avoid vandalism, etc. Being 08:11 jordi part of one of the translation teams grants you "write" access to 08:11 jordi every translation for that language in Ubuntu. Still, if you're 08:11 jordi not a member of your language's team, you can still go ahead and 08:11 jordi translate. Your contributions will be also stored in Rosetta's 08:11 jordi database as "suggestions", but won't appear in Ubuntu's language 08:11 jordi packs until a member of the team reviews and validates them. 08:11 jordi Rosetta offers a long list of applications that can be 08:11 jordi translated. Taking the French team as an example, 08:11 jordi 08:11 jordi https://launchpad.net/distros/ubuntu/edgy/+lang/fr 08:12 jordi we can have a look at how their translation status is for the 08:12 jordi Ubuntu Edgy release. I like showing the French team because they 08:12 jordi are really an amazing example of completeness. 08:14 jordi Rosetta presents us a list of applications which are ready to be 08:14 jordi translated to French, and their current translation status. As 08:14 jordi you see, the French have done their homework and there's barely 08:14 jordi no red bars, meaning "untranslated". See the bottom of the 08:14 jordi page for the meaning of the bar colours. 08:15 jordi The list is ordered from most important to less important 08:15 jordi Let's see how we'd translate an application. Close to the top of 08:16 jordi the list is "launchpad-integration". We'll pick this one as it's 08:16 jordi easy and short. 08:16 jordi https://translations.launchpad.net/distros/ubuntu/edgy/+source/launchpad-integration/+pots/launchpad-integration/fr/+translate 08:17 jordi If instead of French you want to have a look at your own 08:17 jordi language's translation, simply replace "/fr" in the URL with the 08:17 jordi corresponding ISO 639 code. You can find the code for your 08:17 jordi language here: 08:17 jordi http://www.loc.gov/standards/iso639-2/php/code_list.php 08:18 jordi In our case, the first string is "The Launchpad helper 08:18 jordi application failed", which is already translated to French as 08:18 jordi "L'assistant Launchpad a chou". Below the accepted translation 08:18 jordi there is a list of alternative translations suggested by other 08:18 jordi people. You can quickly navigate through the translation fields 08:18 jordi using the tab key. Once you have completed all the strings in a 08:18 jordi page, you want to save your work: hit "Save & Continue" at the 08:18 jordi bottom, and if there are more strings to do, Rosetta will then 08:18 jordi show them to you. 08:19 jordi There are other bits that can help the translators while they 08:19 jordi work on a translation: you might want to see what the translators 08:19 jordi to a language similar to yours used in a string that is hard to 08:19 jordi translate, for inspiration. You can get such information using 08:19 jordi the "Make suggestions from" widget at the top of the string list. 08:19 jordi Also, you'll be more interested in seeing the strings that need 08:19 jordi work instead of those which are translated already. You can 08:19 jordi filter the kind of messages you want to see using the "Show" 08:19 jordi widget, where you can select from "all", "unstranslated", 08:19 jordi "translated" and "needs review".
< neophile> Is it possible to search for a string in the translation? That's major drawback when trying to correct translations. Is a search feature planned for rosseta in the near future?
- It's currently not possible to search for a string easily. it is one of the most requested (and no doubtely most useful) features, and we do plan to add it. the implementation isn't trivial though, as the database is huge and there are some perdformance issues to solve. But yes, the team will focus on providing it as soon as possible
08:22 jordi Using Rosetta's import/export interface 08:22 jordi While the web interface has allowed many Ubuntu users help out 08:22 jordi with the translations to their language, there's certainly 08:22 jordi die-hard, old-time translators who will prefer using their own 08:22 jordi tools (obscure emacs modules and weird command line tools!) to 08:22 jordi work on their translations. Or there might be people who cannot 08:22 jordi afford to be online during the whole translating session. 08:23 jordi To help them, Rosetta has an import/export mechanism, which 08:23 jordi allows you to easily upload translations you have worked on 08:23 jordi offline, using your own ways, but you still want to see 08:23 jordi integrated in Rosetta, and download your finalised files so you 08:23 jordi can do whatever you want with them: back them up, send them to 08:23 jordi your team's mailing list, send them to the upstream author so 08:23 jordi they get included in the next release... 08:24 jordi Importing and exporting is easy: to download your work, use the "Download" and "Upload file" links in the boxes at the left side 08:24 jordi When requesting a download, Launchpad will prepare the file for you and will email you the location of the desired export. 08:25 jordi Importing is similar. Just fill in the field with the location path to your file, and rosetta will integrate it in the database 08:26 jordi I translated all night long. What now? 08:26 jordi ====================================== 08:26 jordi Okay, so you've worked on the files you were interested in, and 08:26 jordi Rosetta now has all the info. What happens now? 08:26 jordi Ubuntu will, on a monthly basis, extract all the translations 08:26 jordi from the database and put them in the "language packs" for each 08:26 jordi supported language in the distro, which will automatically hit 08:26 jordi your Ubuntu mirror the 1st Monday of the month. This way, Rosetta 08:26 jordi allows people to keep improving the support for their language 08:26 jordi even after a Ubuntu release has shipped. For example, more than 6 08:26 jordi months after the release of Ubuntu 6.06 LTS, there's a group 08:26 jordi working on adding Dzonghka support to Ubuntu, when there was 08:26 jordi close to nothing included in dapper initially. 08:27 jordi I see there's many interesting questions going on, so I propose we go on with Q+A, trying to focus on the classics: Rosetta and upstream relationship, etc.
< bugman> In https://translations.launchpad.net/people/bugman/+translations for example, is possible to implements the view of all string translated for a package and not only the last?
- it's possible, yes. These are wishlist features, though, and will get a lower priority than say "search a string" but the info is in the database, so it's prefectly possible to show the info. it's important that bugs are filed against rosetta requesting these things
< dneary> What's the recommended workflow for updating .po files outside Launchpad for the moment?
- can you be more specific? there's several scenarios: gnome-panel or launchpad-integration, for example
[ivoks] he asks if he could see all his translations in one package, just his own, or someone elses.
[dneary] It's related to bug #68014 It seems like uploading .po files doesn't automatically update translations at the moment (if I understand the problem) right, the import mechanism is restricted right now, while a nasty bug involving reverted translations is tracked down
[dneary] Some translations were lost, so on the 1st of November, the upload form was disabled.
for the time being, mailing rosetta@launchpad.net with import requests is the recommended way. although I hope we'll go back to normal operation rsn -- aiui the bug fixing is making progress
< dand> any packages in Rosetta that sync automatically with upstream? if yes, are they marked somehow in Launchpad and how often do they sync?
- good one: this doesn't happen right now, but is a desired feature. this would allow minimise the "conflicts" between rosetta and upstream preojects (KDE, GNOME...) translations, so if some translator has rights to translate both on Ubuntu and GNOME CVS, a translation inserted in Rosetta could be exported to GNOME. we want this, but it'll won't be here before some time
< Gwaihir> is it possible to have a --use-fuzzy implementation for exporting mo files?
- as far as I know, we don't do this now because Rosetta generates fuzzies using its own knowledge. I guess it'd be easy to add an option to generate files with fuzzies, yes. Is there a bug filed?
< Gwaihir> is there any difference between "fuzzy string" in po and "Need review" in Rosetta?
- I've gone over this as I wanted to keep the text dump simple, but yeah, needs review can map to fuzzy. some teams do use it as in the strict "needs review" sense though. ie, theey translate it, but if they are not sure, they use the mark so others can easily find the unsure strings. [danilos] another thing to note: currently, both fuzzy and string needing review is implemented using the same mechanism in rosetta; there is, however, a plan to separate these out and put them to their right meanings (i.e. fuzzy == machine-selected "similar" string; needs review == human selected "unsure" string)
< somerville32> How does a member of a translation team approve suggested translations?
The current implementation is built on very complex copy and paste technology
we do have plans to have the checkbox I'm a bit on time pressure I guess, otherwise I'd dig the relevant spec urls
< somerville32> If I suggest a translation and then join the translation team later, are all my suggested translations automatically approved?
- no, as far as I know. That could be dangerous on some cases, actually, but it might be a good idea to do it if the string was unstranslated.
[somerville32] So I have to go and redo all my work? haha I can't stress enough that having bug reports for all the requests is really helpful
< dand> any plans so far on opening Rosetta for contributions?
- "opening", I assume you mean opening the source code. There's sabdfl's statement that Launchpad will be freed when the project is ready to do so. We can't give dates or estimations on when that might happen. There are people helping out with Launchpad on a NDA basis, though.
[dneary] Does Rosetta keep a history of translation updates and who made them?
- yes, Rosetta keeps a history of all translation contributions. [danilos] dneary: but no, we don't have the level of details you wonder about (when string has moved from suggestion to approved, etc.)
< bugman> It's possibile to hava a Wiki page (or other similar) for see LangPack scheduling?
- the general rule is "1st Monday of the month". We chose this because it's easy to remember. I'm sure this is written up somewhere though, can't find a pointer irght now. there's a Plan to write docs pointing these things clearly
< dneary> Does Rosetta keep a history of translation updates and who made them?
- a feature showing how a string has changed over time has just been rolled onto production, and Rosetta is now gathering this info. This will be good to help team leaders track bad translators
< neophile> Where someone should translate a package, in head brunch or in the edgy brunch?
- this very much depends on the specific packages. we have a feature on our queue, which will allow someone translating gnome-panel in edgy "push" that same translation to dapper, feisty, or GNOME CVS HEAD if applicable, so you just have to translate once. of course, if you translate the Panel in GNOME CVS, that translation will automagically percolate to the next ubuntu release. this gets us to the "working with upstream" chapter, while rosetta is a great tool to get ubuntu translated verfy easily, it's vital that the translation teams cooperate with their upstream teams. ie, the French Ubuntu team should be in contact with the French GNOME team, so they use the same guidelines, etc or don't duplicate efforts. we've had problems with some teams redoing all the work in Rosetta, which resulted in a completely different set of translations in Ubuntu and other distros. The "translation override" feature in rosetta is powerful and useful if used wisely. If it goes out of control, it can cause frictiuons between teams. We need to work on that, I believe it's a procedural problem which can be mostly solved by educating new translators which join the ubuntu teams. there's been quite some talk on this on the rosetta list, and teams like italian or Brazilian have already implemented some measures to work nicely with their upstreams
< aleka> If I am eager to help the Ubuntu community, and the only way I can right now if through translation, What Can I do when the admin of a team does not respond to my requests or approve my membership to the team?
we're seeing this every now and then. the best and quickest way is to mail us at rosetta at launchpad.net and we'll mediaate. ie, we'll try to contact the current leader. If he doesn't respont, we'll transfer leadership to whoever we think deserves it (ie, whoever wants to do the work
[pepsiman] lots of people apply to teams without telling the team leader their email address
<dneary> We have had some issues with translations where people outside the project translated things badly, and we couldn't easily revert to the correct translation - how can I configure our translation to make sure only approved translation team members can do the translation?
- [danilos] at the moment, members of the translation team should only be those you trust; in other words, anyone who is member of your translation team can approve/modify translations. right, we ACK that there's oversized teams right now. [danilos] dneary: yes; anyone else can post suggestions -- we'll work on improving team layout (and docs), so this is clear to everybody
[dneary] And only translation team members can do translations?
- if the given project is setup up like that, yes
< tonyyarusso> Does Rosetta have some method for handling dialects? I'm just getting the ball rolling for a language (oji) that varies widely (esp. for technical terms)
- not yet, but both danilos and I are involved with teams which would benefit of dialect support. We really want to see this happen, but it's not high priority right now.
< somerville32> How do you know if a translation is suggested or approved?
- a approved translation appears in the translation field. suggestions appear as suggestions beneath it [danilos] any actual translation is the approved one; suggestions only appear as "suggestions"
09:01 tonyyarusso jordi: What is the best way to approach things for the time being then until it is implemented?
< dneary> Can I export translations in other formats than .po?
no, Rosetta currently only groks po files. although it will soon be able to export firefox and openoffice data via langpacks but not via the standard export mechanism as far as I know, danilos can correct me. btw, we're working on adding native support for other formats right now; first to come should be firefox XPIs, OpenOffice GSI's, KDE PO files, but we also have plans for XLIFF and others
< dneary> What happens in Rosetta when an overridden string gets changed in a later revision of the upstream translation?
- Rosetta has a "tracker" which makes it think it should follow what comes from upstram or not. In short, if you change a string, forking it to something different, it'll stay forked unless you put the upstram string back again.
[dneary] How do you get to decide?
- We want to implement a filter so it's easy to find these strings and do a "merge with upstream" for them, etc.
[dneary] Is there a fuzziness thing that shows you changed over-ridden strings?
- no, there's proposals to mark them as such
< aleka> where can I get help in aquiring fonts that I need for translating (Amharic - ethiopic fonts) that work in Linux. This question emailed to team leader twice and no response..