ReleaseCycle

Differences between revisions 20 and 24 (spanning 4 versions)
Revision 20 as of 2005-04-29 01:15:24
Size: 10143
Editor: intern146
Comment: BoF notes
Revision 24 as of 2008-08-06 16:36:33
Size: 7701
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from UbuntuDownBelow/BOFs/ReleaseCycle
## page was renamed from ReleaseCycle
## page was renamed from UbuntuDownUnder/BOFs/UbuntuDevelopment/ReleaseCycle
Line 3: Line 6:
= TranslationProcess = = Release Cycle =
Line 7: Line 10:
 * Created: [[Date(2005-04-26T00:56:15Z)]] by JaneW
 * Priority: NeedsPriority
 * People: AdiAttarLead, MatthiasKloseSecond
 * Contributors: JaneW
 * Interested: MartinPitt, CarlosPerelloMarin, DafyddHarries
 * Status: BrainDump, BreezyGoal, UduBof, NewSpec
 * Branch:
 * Malone Bug:
 * Packages:
 * Depends:
 * Dependents:
 [[FullSearch(TranslationProcess)]]
 * UduSessions: 1
  * Created: <<Date(2005-04-23T11:46:01Z)>> by MattZimmerman<<BR>>
  * Priority: MediumPriority<<BR>>
  * People: MattZimmermanLead, ColinWatsonSecond<<BR>>
  * Contributors: MattZimmerman, ColinWatson, AdiAttar, JordiMallach<<BR>>
  * Interested: JorgeBernal, JeffWaugh<<BR>>
  * Status: DraftSpec, BreezyGoal, UduBof, DistroSpecification, MattZimmermanQueue<<BR>>
  * Packages: <<BR>>
  * Depends: <<BR>>
  * UduSessions: 1<<BR>>
Line 23: Line 22:
Translation of open source software has traditionally been a collection of community efforts. One of Canonical's focuses is to offer a complete translation of the Ubuntu desktop as a service. A pilot project, translating Ubuntu into the South African language Xhosa, was carried out from January - March 2005. The learning from this pilot project as well as requirements and recommendations for future translation projects are discussed in this document. Review our experiences with the existing release strategy and discuss possible improvements.
Line 27: Line 26:
In order to offer professional translation services, processes need to be agreed upon and enforced to ensure that translations can be completed on time for the release for which it is intended. The open source landscape is extremely dynamic, which results in constant fluctuation of the translation work to be done for a particular release. The purpose of this exercise is to determine an implementable process that can provide:

 * reliable translation work estimates (for planning and budgeting purposes)
 * a reliable and uptodate source of Ubuntu translation templates
 * enforceable freeze dates for a more stable and predictable translation environment
 * additional tools to aid translation (e.g. profiling and validation)
 * smooth integration with Rosetta for offline translating
Our current release strategy has worked pretty well, but it has encountered some teething problems as new developers, documenters, translators, and so on have joined our community. We need to review and update it.
Line 37: Line 30:
The outcomes of this process will be useful for translation projects of future Ubuntu releases, both commercial and community-based.
Line 41: Line 32:
Each of the points mentioned in the Rationale is discussed separately in this section. At the moment, our release cycle looks like this:
Line 43: Line 34:
=== Reliable Translation Work Estimates ===  * T-18 weeks: seed freeze
 * T-13 weeks: upstream version freeze
 * T-12 weeks: remaining merges complete
 * T-8 weeks: feature freeze
 * T-5 weeks: preview freeze
 * T-4 weeks: preview release
 * T-1 week: final freeze, release candidate
 * T: release
Line 45: Line 43:
For the purposes of planning and budgeting, it is necessary to have a method for obtaining reliable estimates of the amount of work to be done. Typically, translation estimates are given as a number of ''strings'', where strings can have any number of words. Also, typically, professional translation companies manage their quotes and planning based on the number of ''words''. The Hoary release cycle exposed a few problems with this arrangement. Notably, there is no string freeze, and thus translators had problems keeping up to date; there is no point where we freeze the user interface that needs to be documented; we do not adequately notify relevant people of freeze exceptions; and the time allocated between feature freeze and the preview release is a little too short to allow new features to settle down.
Line 47: Line 45:
Currently, getting an accurate estimate for the number of strings is tricky because, although it is theoretically possible to generate a list of packages that officially constitute the desktop, the versions of packages that will be used in the release may change over time, as well as individual strings in each package (strings are added, removed, or changed over time). In the pilot project, this was dealt with by adding an error margin of about 20%. We propose the following changes to the release cycle:
Line 49: Line 47:
Estimating the number of words, rather than strings, adds an extra complication that not all words should be included in the estimate (such as newline characters and variable names). It was suggested that we should work on an automated way of generating a reasonable estimate for the number of words to be translated, taking into heuristics for ignoring non-translatable words, including:  * Remove seed freeze
Line 51: Line 49:
 * escaped characters (newlines, tabs, etc.)
 * variables (appear differently in Gnome, OOo, Mozilla)
 * xml code
 * pathnames (everything that starts with a slash until the next whitespace: /\/\S+/
 The usefulness of the seed freeze has proven to be limited. In practice, we have made necessary seed changes quite late in the cycle, and we always review the supportability of new packages in main regardless of when they are introduced. Instead of an artificial and false seed freeze, we will instead simply take other freeze conditions (upstream version freeze, feature freeze, etc.) into account when making seed changes. This codifies existing practice.
Line 56: Line 51:
MartinPitt will develop some scripts which regularly produce and publish translation statistics on `http://people.ubuntu.com/~pitti/translation-statistics/`.  * Move UVF and feature freeze one week earlier
Line 58: Line 53:
=== Official Source for Ubuntu Translations ===  The gap between feature freeze and the preview release is too short, and a number of new features did not receive adequate testing before the Hoary preview. To alleviate the rush, we will move feature freeze one week earlier. Upstream version freeze will move one week earlier to correspond with this: in practice, we have been content to allow reasonable exceptions to the upstream version freeze, so this should not be a problem.
Line 60: Line 55:
During the pilot project, Rosetta was still in development, so there was no official repository for Ubuntu translations. This resulted in mismatched translations as POT files were obtained from upstream sources. In addition, some Ubuntu-specific strings were not translated.  * Institute string freezes
Line 62: Line 57:
For future translation projects, Rosetta should be considered as the official repository of Ubuntu translation templates. This will ensure that templates:
 * are uptodate with respect to the Ubuntu sources
 * include Ubuntu-specific strings (by regenerating POT files during builds, see LanguagePackRoadmap)
 After consultation with some Ubuntu translators, we will institute a string freeze, to coincide with preview freeze (T-5 weeks). At this point, no translatable strings outside documentation may change without prior confirmation, and any exceptions must be communicated to translators. This should allow translators time to finish their work before the final release.
Line 66: Line 59:
In addition, Rosetta should include non-gettext packages (such as OpenOffice.org and the Mozilla products). This future functionality is planned in LanguagePackRoadmap and OpenOfficeLocalisation.  Since documentation needs to be updated to cope with code changes, we need to allow it a little more time to come up to date, but it nevertheless needs to be translated. We will institute a documentation string freeze, to coincide with the preview release (T-4 weeks). At this point, no translatable strings in the documentation may change without prior confirmation, and any exceptions must be communicated to translators.
Line 68: Line 61:
=== Enforceable Freeze Dates ===  Translations for the installer and other packages not covered by language packs (e.g. `openoffice.org`) must be complete by T-2 weeks. Translations for packages covered by language packs must be complete by T-1 week if they are to be included in the final release; failing that, they will be included in post-release updates.
Line 70: Line 63:
Successful translation requires that freeze dates are planned and enforced. Learning from the Gnome Translation Project suggests that the three milestones of ''Upsteam version freeze'', ''String notification period'' and ''String freeze'' are critical to a translation effort. The Ubuntu release schedule currently caters for an upstream version freeze date. However, for the Hoary release, this freeze was not properly enforced.  * Institute UI freeze
Line 72: Line 65:
In ReleaseCycle it was agreed to:
 * Introduce an Ubuntu string freeze five weeks before the final release.
 * Introduce a string freeze of the English help documents three weeks before the final release.
 * Introduce an Ubuntu string change annoucement period coinciding with the feature freeze.
 * Perform automatic string change detection for string freeze violation checking and automated string change notification.
 Documenters need to have a stable user interface against which to write end-user documentation. To allow for this, we will institute a user interface freeze, one week before preview freeze (T-6 weeks), which by the GNOME 2.10 schedule would have been one week after the GNOME UI freeze. At this point, changes affecting the system's user interface require prior confirmation, and any exceptions must be communicated to translators. Assuming that documentation has been kept reasonably up to date with major changes throughout the cycle, this allows two weeks for further updates before the documentation string freeze.
Line 78: Line 67:
=== Additional Translation Tools ===  * Kernel freeze
Line 80: Line 69:
The following additional tools were suggested:  Changes to the kernel require significant user testing. Testing by the development team often fails to uncover problems that only occur on unusual hardware. At the same time, we often need to make late changes to the kernel in order to fix security vulnerabilities.
Line 82: Line 71:
 1. Automatic system that compares the two most recent set of translatable strings of packages and notifies the uploader about any changes. The upload should then decide whether to revert the change or notify the translation and documentation team. (assigned to MartinPitt).  We will stop making changes to the kernel other than security fixes at T-2 weeks. Any non-security changes after this date will require prior confirmation.
Line 84: Line 73:
 2. Profiling of applications to determine the translation priority of strings, i.e.
    * String that constitute the UI
    * Strings that are rarely seen (a result of rare error cases, etc.)
    * Anything inbetween
 * Communication
Line 89: Line 75:
 3. An Ubuntu glossary. The pilot project made use of the Gnome glossary, which was found to be fairly useful and reasonably sized. A glossary for Ubuntu is a planned part of Rosetta development and this should naturally improve Ubuntu translations.  Developers, documenters, translators, and other interested parties all find it useful to know about significant events in the release cycle in advance, and to be notified of exceptions so that they can take account of them in their areas of responsibility.
Line 91: Line 77:
 4. Adding more validation checks to strings in Rosetta (such as tests available in the translate toolkit):
    * double-spacing
    * inconsistent punctuation (periods, commas, colons, brackets, quotes, etc.)
    * acronyms
    * accelerators
 Rosetta currently supports hard error checking (TranslationValidation spec) -- this includes certain syntax and variable checks. The suggestion is that the additional checks could appear as warnings.
 We will notify relevant mailing lists of significant events in the release cycle one week in advance. If possible (MartinPitt is investigating), we will notify translators automatically of string changes from the feature freeze onwards. We will create a mailing list (name TBD - ubuntu-release?) where freeze exceptions are requested and approved or rejected; this formalises the current IRC-based process, and allows those people not on `#ubuntu-devel` to find out about exceptions and why they were made.
Line 98: Line 79:
=== Offline Translating ===  * Release timing
Line 100: Line 81:
Offline translation is especially important to consider in countries that are
bandwidth challenged and situations where translators are not always connected
to the Internet. The pilot project showed that there are sufficient tools to
support translation offline (for both Linux and Windows platforms). The main issue
is regarding the integration of the translations with Rosetta.
 * PO files should be exportable and importable (currently supported)
 * POT files should be exportable (''not'' currently supported)
 * It was suggested that a soft-locking mechnism (similar to wiki-functionality) be added to Rosetta to allow translators to ''check out'' a PO file for a number of hours or days at a time
 * Another suggestion raised was to develop a custom Rosetta PyGtk application for offline translating
 It is desirable to avoid uploads on the day of release even for feature goals. Therefore, since GNOME releases take place on Wednesdays, we will plan our releases to take place on Thursdays, and allow this to slip to Friday if absolutely necessary.
Line 110: Line 83:
TBC: Decision on way forward Following these changes, the release cycle looks like this:

 * T-14 weeks: upstream version freeze
 * T-13 weeks: remaining merges complete
 * T-9 weeks: feature freeze
 * T-6 weeks: UI freeze
 * T-5 weeks: preview freeze, string freeze
 * T-4 weeks: preview release, doc string freeze
 * T-2 weeks: artwork, installer, non-langpack translations, kernel non-security
 * T-1 week: langpack translations, release candidate
 * T: release

=== Data Preservation and Migration ===

=== Packages Affected ===

=== User Interface Requirements ===
Line 114: Line 103:
Should release announcements be written well in advance so that they can be translated? (It can spoil the surprise if we write them too early.)
Line 116: Line 107:
 * Prerelease dependencies and timelines
 * Process and freeze dates for:
   * Documentation
   * Installer
   * Ubuntu packages
 * Integration and synchronization issues:
   * Official sources
   * Lag between upstream and Ubuntu
   * Ubuntu-specific changes
   * Ubuntu-specific strings
 * Translating offline (and integration with Rosetta)
 * Everybody loves newer software
  * The backports mess
  * What are the valid use cases, and can we accomodate them within Ubuntu?
   * Making new packages available to users of stable releases
   * Making new versions of existing packages available to users of stable releases (at what granularity?)
 * Phases of freeze
 * Release rollup / checklist items
  * Artwork
  * Translations
   * String freeze
   * Installer translations
   * Non-langpack translations
   * Langpack translations
  * Documentation
   * Screenshots
   * Translations?
   * Behaviour changes
  * Kernel
  * Uploads of packages which go on the CD
  * Uploads to main
  * Uploads to universe
Line 128: Line 129:
=== BoF notes (to be sorted and spec'ed out) ===

Main issues:

 * Better estimation of work
 * String freeze violation checking/notification
 * Better string validation/warnings
 * Prioritizing strings within a package (profiling)
 * Release management

Notes to be sorted out:

 * Statistics needed:
    * Number of strings in the desktop release
    * Number of words
    * Number already translated, number fuzzy
 * Translation with nonprofessionals: lack of technical terms, lack of vocab variety -> provide glossary
 * South Africa: no bandwith -> Rosetta is not appropriate there
   * Offline apps: Emacs PO, kbabel
   * Custom Rosetta PyGtk application?
 * String freezes
   * Same time as PreviewFreeze
   * Nice for future: official Ubuntu template with proper enforced freeze days rather than translating upstream CVS
   * Breaking freezes 'really' hurts and breaks translations
   * Enforce string freeze at the time of feature freeze
   * Automatically check string freeze violations after that time
     * Script integrated with buildd
     * Catch accidental violations
   * Ubntu specific strings will appear in Rosetta (-> regenerate POT files in builds, see LanguagePackRoadmap)
   * Need to talk with MDZ and Colin W about release management
   * Need to coordinate with the doc team
      * The doc team is dependent on feature freeze, screenshots not changing
 * Integrate FireFox and OpenOffice strings into gettext/Rosetta translation system
 * Separate the set of translatable strings into important and unimportant strings
   * Proposal: wrapper around gettext() which counts string usage, separate UI and unimportant corner case strings
   * Count how many different PO templates a string appears in?
 * Error checking
   * Hard errors
     * Syntax (missing quote marks)
     * Interpolations (%s)
   * Soft errors
     * Double spaces
     * Accelerators
   * Rosetta now supports hard error checking (TranslationValidation spec) -- we should look at supporting warnings also.
   * Need to customise error checking on a per-language basis
=== UDU Pre-Work ===

Release Cycle

Status

Introduction

Review our experiences with the existing release strategy and discuss possible improvements.

Rationale

Our current release strategy has worked pretty well, but it has encountered some teething problems as new developers, documenters, translators, and so on have joined our community. We need to review and update it.

Scope and Use Cases

Implementation Plan

At the moment, our release cycle looks like this:

  • T-18 weeks: seed freeze
  • T-13 weeks: upstream version freeze
  • T-12 weeks: remaining merges complete
  • T-8 weeks: feature freeze
  • T-5 weeks: preview freeze
  • T-4 weeks: preview release
  • T-1 week: final freeze, release candidate
  • T: release

The Hoary release cycle exposed a few problems with this arrangement. Notably, there is no string freeze, and thus translators had problems keeping up to date; there is no point where we freeze the user interface that needs to be documented; we do not adequately notify relevant people of freeze exceptions; and the time allocated between feature freeze and the preview release is a little too short to allow new features to settle down.

We propose the following changes to the release cycle:

  • Remove seed freeze The usefulness of the seed freeze has proven to be limited. In practice, we have made necessary seed changes quite late in the cycle, and we always review the supportability of new packages in main regardless of when they are introduced. Instead of an artificial and false seed freeze, we will instead simply take other freeze conditions (upstream version freeze, feature freeze, etc.) into account when making seed changes. This codifies existing practice.
  • Move UVF and feature freeze one week earlier The gap between feature freeze and the preview release is too short, and a number of new features did not receive adequate testing before the Hoary preview. To alleviate the rush, we will move feature freeze one week earlier. Upstream version freeze will move one week earlier to correspond with this: in practice, we have been content to allow reasonable exceptions to the upstream version freeze, so this should not be a problem.
  • Institute string freezes After consultation with some Ubuntu translators, we will institute a string freeze, to coincide with preview freeze (T-5 weeks). At this point, no translatable strings outside documentation may change without prior confirmation, and any exceptions must be communicated to translators. This should allow translators time to finish their work before the final release. Since documentation needs to be updated to cope with code changes, we need to allow it a little more time to come up to date, but it nevertheless needs to be translated. We will institute a documentation string freeze, to coincide with the preview release (T-4 weeks). At this point, no translatable strings in the documentation may change without prior confirmation, and any exceptions must be communicated to translators.

    Translations for the installer and other packages not covered by language packs (e.g. openoffice.org) must be complete by T-2 weeks. Translations for packages covered by language packs must be complete by T-1 week if they are to be included in the final release; failing that, they will be included in post-release updates.

  • Institute UI freeze Documenters need to have a stable user interface against which to write end-user documentation. To allow for this, we will institute a user interface freeze, one week before preview freeze (T-6 weeks), which by the GNOME 2.10 schedule would have been one week after the GNOME UI freeze. At this point, changes affecting the system's user interface require prior confirmation, and any exceptions must be communicated to translators. Assuming that documentation has been kept reasonably up to date with major changes throughout the cycle, this allows two weeks for further updates before the documentation string freeze.
  • Kernel freeze Changes to the kernel require significant user testing. Testing by the development team often fails to uncover problems that only occur on unusual hardware. At the same time, we often need to make late changes to the kernel in order to fix security vulnerabilities. We will stop making changes to the kernel other than security fixes at T-2 weeks. Any non-security changes after this date will require prior confirmation.
  • Communication Developers, documenters, translators, and other interested parties all find it useful to know about significant events in the release cycle in advance, and to be notified of exceptions so that they can take account of them in their areas of responsibility.

    We will notify relevant mailing lists of significant events in the release cycle one week in advance. If possible (MartinPitt is investigating), we will notify translators automatically of string changes from the feature freeze onwards. We will create a mailing list (name TBD - ubuntu-release?) where freeze exceptions are requested and approved or rejected; this formalises the current IRC-based process, and allows those people not on #ubuntu-devel to find out about exceptions and why they were made.

  • Release timing It is desirable to avoid uploads on the day of release even for feature goals. Therefore, since GNOME releases take place on Wednesdays, we will plan our releases to take place on Thursdays, and allow this to slip to Friday if absolutely necessary.

Following these changes, the release cycle looks like this:

  • T-14 weeks: upstream version freeze
  • T-13 weeks: remaining merges complete
  • T-9 weeks: feature freeze
  • T-6 weeks: UI freeze
  • T-5 weeks: preview freeze, string freeze
  • T-4 weeks: preview release, doc string freeze
  • T-2 weeks: artwork, installer, non-langpack translations, kernel non-security
  • T-1 week: langpack translations, release candidate
  • T: release

Data Preservation and Migration

Packages Affected

User Interface Requirements

Outstanding Issues

Should release announcements be written well in advance so that they can be translated? (It can spoil the surprise if we write them too early.)

UDU BOF Agenda

  • Everybody loves newer software
    • The backports mess
    • What are the valid use cases, and can we accomodate them within Ubuntu?
      • Making new packages available to users of stable releases
      • Making new versions of existing packages available to users of stable releases (at what granularity?)
  • Phases of freeze
  • Release rollup / checklist items
    • Artwork
    • Translations
      • String freeze
      • Installer translations
      • Non-langpack translations
      • Langpack translations
    • Documentation
      • Screenshots
      • Translations?
      • Behaviour changes
    • Kernel
    • Uploads of packages which go on the CD
    • Uploads to main
    • Uploads to universe

UDU Pre-Work

UbuntuDownUnder/BOFs/ReleaseCycle (last edited 2008-08-06 16:36:33 by localhost)