ReleaseCycle

Differences between revisions 18 and 20 (spanning 2 versions)
⇤ ← Revision 18 as of 2005-04-28 09:25:59 →
Size: 7591
Editor: intern146
Comment: fix bulleting
← Revision 20 as of 2005-04-29 01:15:24 → ⇥
Size: 10143
Editor: intern146
Comment: BoF notes
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from UbuntuDownUnder/BOFs/UbuntuDevelopment/ReleaseCycle
Line 4: Line 3:
= Release Cycle = = TranslationProcess =
Line 8: Line 7:
  * Created: [[Date(2005-04-23T11:46:01Z)]] by MattZimmerman[[BR]]
  * Priority: MediumPriority[[BR]]
  * People: MattZimmermanLead, ColinWatsonSecond[[BR]]
  * Contributors: MattZimmerman, ColinWatson, AdiAttar, JordiMallach[[BR]]
  * Interested: JorgeBernal[[BR]]
  * Status: DraftSpec, BreezyGoal, UduBof, DistroSpecification, MattZimmermanQueue[[BR]]
  * Packages: [[BR]]
  * Depends: [[BR]]
  * UduSessions: 1[[BR]]		  * Created: [[Date(2005-04-26T00:56:15Z)]] by JaneW
 * Priority: NeedsPriority
 * People: AdiAttarLead, MatthiasKloseSecond
 * Contributors: JaneW
 * Interested: MartinPitt, CarlosPerelloMarin, DafyddHarries
 * Status: BrainDump, BreezyGoal, UduBof, NewSpec
 * Branch:
 * Malone Bug:
 * Packages:
 * Depends:
 * Dependents:
 [[FullSearch(TranslationProcess)]]
 * UduSessions: 1
Line 20: Line 23:
Review our experiences with the existing release strategy and discuss possible improvements. Translation of open source software has traditionally been a collection of community efforts. One of Canonical's focuses is to offer a complete translation of the Ubuntu desktop as a service. A pilot project, translating Ubuntu into the South African language Xhosa, was carried out from January - March 2005. The learning from this pilot project as well as requirements and recommendations for future translation projects are discussed in this document.
Line 24: Line 27:
Our current release strategy has worked pretty well, but it has encountered some teething problems as new developers, documenters, translators, and so on have joined our community. We need to review and update it. In order to offer professional translation services, processes need to be agreed upon and enforced to ensure that translations can be completed on time for the release for which it is intended. The open source landscape is extremely dynamic, which results in constant fluctuation of the translation work to be done for a particular release. The purpose of this exercise is to determine an implementable process that can provide:

 * reliable translation work estimates (for planning and budgeting purposes)
 * a reliable and uptodate source of Ubuntu translation templates
 * enforceable freeze dates for a more stable and predictable translation environment
 * additional tools to aid translation (e.g. profiling and validation)
 * smooth integration with Rosetta for offline translating
Line 28: Line 37:
The outcomes of this process will be useful for translation projects of future Ubuntu releases, both commercial and community-based.
Line 30: Line 41:
At the moment, our release cycle looks like this: Each of the points mentioned in the Rationale is discussed separately in this section.
Line 32: Line 43:
 * T-18 weeks: seed freeze
 * T-13 weeks: upstream version freeze
 * T-12 weeks: remaining merges complete
 * T-8 weeks: feature freeze
 * T-5 weeks: preview freeze
 * T-4 weeks: preview release
 * T-1 week: final freeze, release candidate
 * T: release		 === Reliable Translation Work Estimates ===
Line 41: Line 45:
The Hoary release cycle exposed a few problems with this arrangement. Notably, there is no string freeze, and thus translators had problems keeping up to date; there is no point where we freeze the user interface that needs to be documented; we do not adequately notify relevant people of freeze exceptions; and the time allocated between feature freeze and the preview release is a little too short to allow new features to settle down. For the purposes of planning and budgeting, it is necessary to have a method for obtaining reliable estimates of the amount of work to be done. Typically, translation estimates are given as a number of ''strings'', where strings can have any number of words. Also, typically, professional translation companies manage their quotes and planning based on the number of ''words''.
Line 43: Line 47:
We propose the following changes to the release cycle: Currently, getting an accurate estimate for the number of strings is tricky because, although it is theoretically possible to generate a list of packages that officially constitute the desktop, the versions of packages that will be used in the release may change over time, as well as individual strings in each package (strings are added, removed, or changed over time). In the pilot project, this was dealt with by adding an error margin of about 20%.
Line 45: Line 49:
 * Remove seed freeze Estimating the number of words, rather than strings, adds an extra complication that not all words should be included in the estimate (such as newline characters and variable names). It was suggested that we should work on an automated way of generating a reasonable estimate for the number of words to be translated, taking into heuristics for ignoring non-translatable words, including:
Line 47: Line 51:
 The usefulness of the seed freeze has proven to be limited. In practice, we have made necessary seed changes quite late in the cycle, and we always review the supportability of new packages in main regardless of when they are introduced. Instead of an artificial and false seed freeze, we will instead simply take other freeze conditions (upstream version freeze, feature freeze, etc.) into account when making seed changes. This codifies existing practice.  * escaped characters (newlines, tabs, etc.)
 * variables (appear differently in Gnome, OOo, Mozilla)
 * xml code
 * pathnames (everything that starts with a slash until the next whitespace: /\/\S+/
Line 49: Line 56:
 * Move UVF and feature freeze one week earlier MartinPitt will develop some scripts which regularly produce and publish translation statistics on `http://people.ubuntu.com/~pitti/translation-statistics/`.
Line 51: Line 58:
 The gap between feature freeze and the preview release is too short, and a number of new features did not receive adequate testing before the Hoary preview. To alleviate the rush, we will move feature freeze one week earlier. Upstream version freeze will move one week earlier to correspond with this: in practice, we have been content to allow reasonable exceptions to the upstream version freeze, so this should not be a problem. === Official Source for Ubuntu Translations ===
Line 53: Line 60:
 * Institute string freezes During the pilot project, Rosetta was still in development, so there was no official repository for Ubuntu translations. This resulted in mismatched translations as POT files were obtained from upstream sources. In addition, some Ubuntu-specific strings were not translated.
Line 55: Line 62:
 After consultation with some Ubuntu translators, we will institute a string freeze, to coincide with preview freeze (T-5 weeks). At this point, no translatable strings outside documentation may change without prior confirmation, and any exceptions must be communicated to translators. This should allow translators time to finish their work before the final release. For future translation projects, Rosetta should be considered as the official repository of Ubuntu translation templates. This will ensure that templates:
 * are uptodate with respect to the Ubuntu sources
 * include Ubuntu-specific strings (by regenerating POT files during builds, see LanguagePackRoadmap)
Line 57: Line 66:
 Since documentation needs to be updated to cope with code changes, we need to allow it a little more time to come up to date, but it nevertheless needs to be translated. We will institute a documentation string freeze, to coincide with the preview release (T-4 weeks). At this point, no translatable strings in the documentation may change without prior confirmation, and any exceptions must be communicated to translators. In addition, Rosetta should include non-gettext packages (such as OpenOffice.org and the Mozilla products). This future functionality is planned in LanguagePackRoadmap and OpenOfficeLocalisation.
Line 59: Line 68:
 Translations for the installer and other packages not covered by language packs (e.g. `openoffice.org`) must be complete by T-2 weeks. Translations for packages covered by language packs must be complete by T-1 week if they are to be included in the final release; failing that, they will be included in post-release updates. === Enforceable Freeze Dates ===
Line 61: Line 70:
 * Institute UI freeze Successful translation requires that freeze dates are planned and enforced. Learning from the Gnome Translation Project suggests that the three milestones of ''Upsteam version freeze'', ''String notification period'' and ''String freeze'' are critical to a translation effort. The Ubuntu release schedule currently caters for an upstream version freeze date. However, for the Hoary release, this freeze was not properly enforced.
Line 63: Line 72:
 Documenters need to have a stable user interface against which to write end-user documentation. To allow for this, we will institute a user interface freeze, one week before preview freeze (T-6 weeks), which by the GNOME 2.10 schedule would have been one week after the GNOME UI freeze. At this point, changes affecting the system's user interface require prior confirmation, and any exceptions must be communicated to translators. Assuming that documentation has been kept reasonably up to date with major changes throughout the cycle, this allows two weeks for further updates before the documentation string freeze. In ReleaseCycle it was agreed to:
 * Introduce an Ubuntu string freeze five weeks before the final release.
 * Introduce a string freeze of the English help documents three weeks before the final release.
 * Introduce an Ubuntu string change annoucement period coinciding with the feature freeze.
 * Perform automatic string change detection for string freeze violation checking and automated string change notification.
Line 65: Line 78:
 * Kernel freeze === Additional Translation Tools ===
Line 67: Line 80:
 Changes to the kernel require significant user testing. Testing by the development team often fails to uncover problems that only occur on unusual hardware. At the same time, we often need to make late changes to the kernel in order to fix security vulnerabilities. The following additional tools were suggested:
Line 69: Line 82:
 We will stop making changes to the kernel other than security fixes at T-2 weeks. Any non-security changes after this date will require prior confirmation.  1. Automatic system that compares the two most recent set of translatable strings of packages and notifies the uploader about any changes. The upload should then decide whether to revert the change or notify the translation and documentation team. (assigned to MartinPitt).
Line 71: Line 84:
 * Communication  2. Profiling of applications to determine the translation priority of strings, i.e.
    * String that constitute the UI
    * Strings that are rarely seen (a result of rare error cases, etc.)
    * Anything inbetween
Line 73: Line 89:
 Developers, documenters, translators, and other interested parties all find it useful to know about significant events in the release cycle in advance, and to be notified of exceptions so that they can take account of them in their areas of responsibility.  3. An Ubuntu glossary. The pilot project made use of the Gnome glossary, which was found to be fairly useful and reasonably sized. A glossary for Ubuntu is a planned part of Rosetta development and this should naturally improve Ubuntu translations.
Line 75: Line 91:
 We will notify relevant mailing lists of significant events in the release cycle one week in advance. If possible (MartinPitt is investigating), we will notify translators automatically of string changes from the feature freeze onwards. We will create a mailing list (name TBD - ubuntu-release?) where freeze exceptions are requested and approved or rejected; this formalises the current IRC-based process, and allows those people not on `#ubuntu-devel` to find out about exceptions and why they were made.  4. Adding more validation checks to strings in Rosetta (such as tests available in the translate toolkit):
    * double-spacing
    * inconsistent punctuation (periods, commas, colons, brackets, quotes, etc.)
    * acronyms
    * accelerators
 Rosetta currently supports hard error checking (TranslationValidation spec) -- this includes certain syntax and variable checks. The suggestion is that the additional checks could appear as warnings.
Line 77: Line 98:
 * Release timing === Offline Translating ===
Line 79: Line 100:
 It is desirable to avoid uploads on the day of release even for feature goals. Therefore, since GNOME releases take place on Wednesdays, we will plan our releases to take place on Thursdays, and allow this to slip to Friday if absolutely necessary. Offline translation is especially important to consider in countries that are
bandwidth challenged and situations where translators are not always connected
to the Internet. The pilot project showed that there are sufficient tools to
support translation offline (for both Linux and Windows platforms). The main issue
is regarding the integration of the translations with Rosetta.
 * PO files should be exportable and importable (currently supported)
 * POT files should be exportable (''not'' currently supported)
 * It was suggested that a soft-locking mechnism (similar to wiki-functionality) be added to Rosetta to allow translators to ''check out'' a PO file for a number of hours or days at a time
 * Another suggestion raised was to develop a custom Rosetta PyGtk application for offline translating
Line 81: Line 110:
Following these changes, the release cycle looks like this:

 * T-14 weeks: upstream version freeze
 * T-13 weeks: remaining merges complete
 * T-9 weeks: feature freeze
 * T-6 weeks: UI freeze
 * T-5 weeks: preview freeze, string freeze
 * T-4 weeks: preview release, doc string freeze
 * T-2 weeks: artwork, installer, non-langpack translations, kernel non-security
 * T-1 week: langpack translations, release candidate
 * T: release

=== Data Preservation and Migration ===

=== Packages Affected ===

=== User Interface Requirements ===		 TBC: Decision on way forward
Line 101: Line 114:
Should release announcements be written well in advance so that they can be translated? (It can spoil the surprise if we write them too early.)
Line 105: Line 116:
 * Everybody loves newer software
  * The backports mess
  * What are the valid use cases, and can we accomodate them within Ubuntu?
   * Making new packages available to users of stable releases
   * Making new versions of existing packages available to users of stable releases (at what granularity?)
 * Phases of freeze
 * Release rollup / checklist items
  * Artwork
  * Translations
   * String freeze
   * Installer translations
   * Non-langpack translations
   * Langpack translations
  * Documentation
   * Screenshots
   * Translations?
   * Behaviour changes
  * Kernel
  * Uploads of packages which go on the CD
  * Uploads to main
  * Uploads to universe		  * Prerelease dependencies and timelines
 * Process and freeze dates for:
   * Documentation
   * Installer
   * Ubuntu packages
 * Integration and synchronization issues:
   * Official sources
   * Lag between upstream and Ubuntu
   * Ubuntu-specific changes
   * Ubuntu-specific strings
 * Translating offline (and integration with Rosetta)
Line 127: Line 128:
=== UDU Pre-Work === === BoF notes (to be sorted and spec'ed out) ===

Main issues:

 * Better estimation of work
 * String freeze violation checking/notification
 * Better string validation/warnings
 * Prioritizing strings within a package (profiling)
 * Release management

Notes to be sorted out:

 * Statistics needed:
    * Number of strings in the desktop release
    * Number of words
    * Number already translated, number fuzzy
 * Translation with nonprofessionals: lack of technical terms, lack of vocab variety -> provide glossary
 * South Africa: no bandwith -> Rosetta is not appropriate there
   * Offline apps: Emacs PO, kbabel
   * Custom Rosetta PyGtk application?
 * String freezes
   * Same time as PreviewFreeze
   * Nice for future: official Ubuntu template with proper enforced freeze days rather than translating upstream CVS
   * Breaking freezes 'really' hurts and breaks translations
   * Enforce string freeze at the time of feature freeze
   * Automatically check string freeze violations after that time
     * Script integrated with buildd
     * Catch accidental violations
   * Ubntu specific strings will appear in Rosetta (-> regenerate POT files in builds, see LanguagePackRoadmap)
   * Need to talk with MDZ and Colin W about release management
   * Need to coordinate with the doc team
      * The doc team is dependent on feature freeze, screenshots not changing
 * Integrate FireFox and OpenOffice strings into gettext/Rosetta translation system
 * Separate the set of translatable strings into important and unimportant strings
   * Proposal: wrapper around gettext() which counts string usage, separate UI and unimportant corner case strings
   * Count how many different PO templates a string appears in?
 * Error checking
   * Hard errors
     * Syntax (missing quote marks)
     * Interpolations (%s)
   * Soft errors
     * Double spaces
     * Accelerators
   * Rosetta now supports hard error checking (TranslationValidation spec) -- we should look at supporting warnings also.
   * Need to customise error checking on a per-language basis

TranslationProcess

Status

Introduction

Translation of open source software has traditionally been a collection of community efforts. One of Canonical's focuses is to offer a complete translation of the Ubuntu desktop as a service. A pilot project, translating Ubuntu into the South African language Xhosa, was carried out from January - March 2005. The learning from this pilot project as well as requirements and recommendations for future translation projects are discussed in this document.

Rationale

In order to offer professional translation services, processes need to be agreed upon and enforced to ensure that translations can be completed on time for the release for which it is intended. The open source landscape is extremely dynamic, which results in constant fluctuation of the translation work to be done for a particular release. The purpose of this exercise is to determine an implementable process that can provide:

  • reliable translation work estimates (for planning and budgeting purposes)
  • a reliable and uptodate source of Ubuntu translation templates
  • enforceable freeze dates for a more stable and predictable translation environment
  • additional tools to aid translation (e.g. profiling and validation)
  • smooth integration with Rosetta for offline translating

Scope and Use Cases

The outcomes of this process will be useful for translation projects of future Ubuntu releases, both commercial and community-based.

Implementation Plan

Each of the points mentioned in the Rationale is discussed separately in this section.

Reliable Translation Work Estimates

For the purposes of planning and budgeting, it is necessary to have a method for obtaining reliable estimates of the amount of work to be done. Typically, translation estimates are given as a number of strings, where strings can have any number of words. Also, typically, professional translation companies manage their quotes and planning based on the number of words.

Currently, getting an accurate estimate for the number of strings is tricky because, although it is theoretically possible to generate a list of packages that officially constitute the desktop, the versions of packages that will be used in the release may change over time, as well as individual strings in each package (strings are added, removed, or changed over time). In the pilot project, this was dealt with by adding an error margin of about 20%.

Estimating the number of words, rather than strings, adds an extra complication that not all words should be included in the estimate (such as newline characters and variable names). It was suggested that we should work on an automated way of generating a reasonable estimate for the number of words to be translated, taking into heuristics for ignoring non-translatable words, including:

  • escaped characters (newlines, tabs, etc.)
  • variables (appear differently in Gnome, OOo, Mozilla)
  • xml code
  • pathnames (everything that starts with a slash until the next whitespace: /\/\S+/

MartinPitt will develop some scripts which regularly produce and publish translation statistics on http://people.ubuntu.com/~pitti/translation-statistics/.

Official Source for Ubuntu Translations

During the pilot project, Rosetta was still in development, so there was no official repository for Ubuntu translations. This resulted in mismatched translations as POT files were obtained from upstream sources. In addition, some Ubuntu-specific strings were not translated.

For future translation projects, Rosetta should be considered as the official repository of Ubuntu translation templates. This will ensure that templates:

  • are uptodate with respect to the Ubuntu sources

  • include Ubuntu-specific strings (by regenerating POT files during builds, see LanguagePackRoadmap)

In addition, Rosetta should include non-gettext packages (such as OpenOffice.org and the Mozilla products). This future functionality is planned in LanguagePackRoadmap and OpenOfficeLocalisation.

Enforceable Freeze Dates

Successful translation requires that freeze dates are planned and enforced. Learning from the Gnome Translation Project suggests that the three milestones of Upsteam version freeze, String notification period and String freeze are critical to a translation effort. The Ubuntu release schedule currently caters for an upstream version freeze date. However, for the Hoary release, this freeze was not properly enforced.

In ReleaseCycle it was agreed to:

  • Introduce an Ubuntu string freeze five weeks before the final release.
  • Introduce a string freeze of the English help documents three weeks before the final release.
  • Introduce an Ubuntu string change annoucement period coinciding with the feature freeze.
  • Perform automatic string change detection for string freeze violation checking and automated string change notification.

Additional Translation Tools

The following additional tools were suggested:

  1. Automatic system that compares the two most recent set of translatable strings of packages and notifies the uploader about any changes. The upload should then decide whether to revert the change or notify the translation and documentation team. (assigned to MartinPitt).

  2. Profiling of applications to determine the translation priority of strings, i.e.
    • String that constitute the UI
    • Strings that are rarely seen (a result of rare error cases, etc.)
    • Anything inbetween
  3. An Ubuntu glossary. The pilot project made use of the Gnome glossary, which was found to be fairly useful and reasonably sized. A glossary for Ubuntu is a planned part of Rosetta development and this should naturally improve Ubuntu translations.
  4. Adding more validation checks to strings in Rosetta (such as tests available in the translate toolkit):
    • double-spacing
    • inconsistent punctuation (periods, commas, colons, brackets, quotes, etc.)
    • acronyms
    • accelerators

    Rosetta currently supports hard error checking (TranslationValidation spec) -- this includes certain syntax and variable checks. The suggestion is that the additional checks could appear as warnings.

Offline Translating

Offline translation is especially important to consider in countries that are bandwidth challenged and situations where translators are not always connected to the Internet. The pilot project showed that there are sufficient tools to support translation offline (for both Linux and Windows platforms). The main issue is regarding the integration of the translations with Rosetta.

  • PO files should be exportable and importable (currently supported)

  • POT files should be exportable (not currently supported)

  • It was suggested that a soft-locking mechnism (similar to wiki-functionality) be added to Rosetta to allow translators to check out a PO file for a number of hours or days at a time

  • Another suggestion raised was to develop a custom Rosetta PyGtk application for offline translating

TBC: Decision on way forward

Outstanding Issues

UDU BOF Agenda

  • Prerelease dependencies and timelines
  • Process and freeze dates for:
    • Documentation
    • Installer
    • Ubuntu packages
  • Integration and synchronization issues:
    • Official sources
    • Lag between upstream and Ubuntu
    • Ubuntu-specific changes
    • Ubuntu-specific strings
  • Translating offline (and integration with Rosetta)

BoF notes (to be sorted and spec'ed out)

Main issues:

  • Better estimation of work
  • String freeze violation checking/notification
  • Better string validation/warnings
  • Prioritizing strings within a package (profiling)
  • Release management

Notes to be sorted out:

  • Statistics needed:
    • Number of strings in the desktop release
    • Number of words
    • Number already translated, number fuzzy

  • Translation with nonprofessionals: lack of technical terms, lack of vocab variety -> provide glossary

  • South Africa: no bandwith -> Rosetta is not appropriate there

    • Offline apps: Emacs PO, kbabel

    • Custom Rosetta PyGtk application?

  • String freezes

    • Same time as PreviewFreeze

    • Nice for future: official Ubuntu template with proper enforced freeze days rather than translating upstream CVS
    • Breaking freezes 'really' hurts and breaks translations
    • Enforce string freeze at the time of feature freeze
    • Automatically check string freeze violations after that time
      • Script integrated with buildd
      • Catch accidental violations

    • Ubntu specific strings will appear in Rosetta (-> regenerate POT files in builds, see LanguagePackRoadmap)

    • Need to talk with MDZ and Colin W about release management
    • Need to coordinate with the doc team
      • The doc team is dependent on feature freeze, screenshots not changing

  • Integrate FireFox and OpenOffice strings into gettext/Rosetta translation system

  • Separate the set of translatable strings into important and unimportant strings
    • Proposal: wrapper around gettext() which counts string usage, separate UI and unimportant corner case strings
    • Count how many different PO templates a string appears in?
  • Error checking
    • Hard errors
      • Syntax (missing quote marks)
      • Interpolations (%s)
    • Soft errors
      • Double spaces
      • Accelerators

    • Rosetta now supports hard error checking (TranslationValidation spec) -- we should look at supporting warnings also.

    • Need to customise error checking on a per-language basis

UbuntuDownUnder/BOFs/ReleaseCycle (last edited 2008-08-06 16:36:33 by localhost)