ReleaseSchedule

Differences between revisions 1 and 19 (spanning 18 versions)
Revision 1 as of 2015-07-07 15:02:17
Size: 4400
Editor: sil2100
Comment:
Revision 19 as of 2016-04-19 16:32:44
Size: 6418
Editor: sil2100
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;"><<TableOfContents>>||
Line 5: Line 7:
Release preparation requires an archive freeze that usually lasts three/four days until the image promotion has happened.
Line 9: Line 10:
 * The Product Team requests an OTA update release every 6 weeks
 * In case another release is required out of the standard timeframe, the Product Team needs to fill in a request at least two weeks before the planned release, announcing it on the mailing lists
 * The Product Team requests an OTA update release every 6 weeks for all officially supported products
 * In case another release is required out of the standard timeframe, the Product Team needs to inform the Landing Team at least two weeks before the planned release, also officially announcing it on the mailing list (OTA point releases)
Line 12: Line 13:
 * During the Release preparation period (see timeline below) the automated Image builds get disabled, silo landings for the targeted distribution get shot down with the exception of regression fixes that were found during image testing. Bugfixes for blocker bugs from the above list have to be in silos '''before''' the freeze.
Line 14: Line 14:
 * During the Release preparation period (see timeline below) a snapshot of the release candidate is made to the snapshot PPA and all re-spins are prepared in the rc channel. The snapshot is updated with cherry-picks from the overlay whenever a regression needs to be fixed.
 * During the Release preparation period normal landings are enabled normally, with rc-proposed images containing the daily changes.
Line 17: Line 20:
=== Tuesday (US TZ) EOD before a Milestone === === Friday (US TZ) EOD (3rd week) ===
Line 19: Line 22:
 * '''All blocking bug fixes''' from the Product team list '''are in silos''' so that they can be signed off by QA when the day in the european timezone starts.  * '''All user visible strings''' are now landed in the image, no silos with additional string changes are allowed to be released to the archive.
Line 21: Line 24:
=== Wednesday morning (EU TZ) before a Milestone === === Friday (US TZ) EOD (4th week) ===
Line 23: Line 26:
 * The '''archive freeze''' gets called out by the LandingTeam, QA tests the final silos in preparation of the Milestone, in case issues are found the developer will work with the QA and Landing teams to get the silo fixed in time for the Milestone release. Please remember that QA has limited resources and if the number of queued up silos is too big, not all of them will be signed-off and landed for this milestone.  * The '''Final Freeze''' gets called out by the LandingTeam. No landings are accepted by the LandingTeam or the QA Team for the selected milestone besides ship blockers and rare ProductTeam approved exceptions.
Line 25: Line 28:
=== Wednesday afternoon (EU TZ) before a Milestone === === Saturday (EU TZ) (4th week) ===
Line 27: Line 30:
 * Once all blocker silos have landed a Milestone '''candidate image is created''', automated image builds get shut down.  * The first automated candidate image is built to the rc-proposed channel.
Line 29: Line 32:
 * The '''QA team starts a regression test''' of the candidate image, in case severe regressions are found, the developer of the component gets contacted and a silo with a fix gets prepared. Once the silo lands a new milestone candidate image is prepared. === Monday morning (EU TZ) (5th week) ===
Line 31: Line 34:
=== Wednesday EOD (US TZ) before a Milestone ===  * The overlay PPA state for the candidate image is copied to the snapshot PPA.
 * The first candidate image is copied from rc-proposed to the rc channel.
 * The '''QA team starts the regression tests''' of the candidate image, in case severe regressions are found, the developer of the component gets contacted and a silo with a fix gets prepared.
 * Landings to the overlay PPA are allowed but limited by QA resources, as QA engineers prioritize release candidate testing over silos.
 * Whenever QA finds a blocking regression and a fix is prepared, the fix is cherry-picked to the snapshot PPA and a new rc image is rebuilt.
Line 33: Line 40:
 * '''Testing''' of the candidate image on the desired target architecture '''is finished''', the LandingTeam is notified about the status. === Friday EOD (US TZ) (5th week) ===
Line 35: Line 42:
=== Thursday morning (EU TZ) ===  * '''Testing''' of the candidate image on the desired target architectures '''is finished''', the LandingTeam is notified about the status.
 * Information about a valid image candidate is sent to the manufacturers for testing.
Line 37: Line 45:
 * The '''QA team runs a quick test on the community architectures''' that will have to be promoted alongside the product image (mako, emulator, etc) and gives a recommendation about releasability of these images to the LandingTeam. === Monday morning (EU TZ) (6th week) ===
Line 39: Line 47:
=== Thursday noon (EU TZ) ===  * The '''QA team runs a quick test on the community architectures''' that will have to be promoted alongside the product image (mako, emulator, etc) and gives a recommendation about realisability of these images to the LandingTeam.
Line 41: Line 49:
 * The '''LandingTeam releases the Milestone''' image based on the recommendations from the QA team (architectures/devices etc) and releases images from the devel-proposed channel of the respective distro to the devel channel (see: LandingTeam/TouchChannels and LandingTeam/SupportedDevices). === Tuesday evening (EU TZ) (6th week) ===
Line 43: Line 51:
 * Automated image builds get enabled again  * The manufacturers return feedback regarding the realisability of the OTA.
 * A final decision is made based off the decisions of QA, Product management, manufacturers feedback and the Landing Team. The Product Team always have the possibility to veto any of the decisions.
Line 45: Line 54:
 * Archive landings get resumed again. === Wednesday noon (EU TZ) (6th week) ===
Line 47: Line 56:
== Technical Details ==  * The '''LandingTeam releases the Milestone'''. The candidate image is copied from RC to stable for all approved channels and devices (ubuntu-touch/stable/*).
 * Phased updates are started for the new stable image.
Line 49: Line 59:
=== Image server operations === === Thursday morning (EU TZ) (6th week) ===
Line 51: Line 61:
En/Disabling automated build (members of the ubuntu-cdimage team):  * Phased updates are over, all users should see update notifications.
Line 53: Line 63:
On the "nusakan" image build machine, become the cdimage user, find your build entry in the crontab and comment it.
{{{
ogra@nusakan:~$ sudo -u cdimage -i
cdimage@nusakan:~$ crontab -e
}}}
== Schedule Overview ==
Line 59: Line 65:
Promoting (members of the ubuntu-cdimage team):
{{{
ogra@nusakan:~$ sudo -u cdimage -i
cdimage@nusakan:~$ /srv/system-image.ubuntu.com/bin/copy-image ubuntu-touch/ubuntu-rtm/14.09-proposed ubuntu-touch/ubuntu-rtm/14.09 krillin 59
}}}
The above shows the promotion command used to release the proposed krillin image 59 into the 14.09 channel on the "nusakan" image build machine. The command is called in a way that the image version in the target channel will automatically be incremented based on the last image in this channel.
||<rowbgcolor="#cccccc"> '''Week''' || '''Day''' || '''''Date for current OTA''''' || '''Events''' ||
||<#CCFFCC> '''1''' || ~Friday || || <!> Milestone bug list announced ||
||<#FFFFCC> '''3''' || Tuesday || || /!\ FeatureFreeze, /!\ StringFreeze ||
||<#FFEBBB> '''4''' || Friday || || /!\ FinalFreeze, /!\ ReleaseCandidate ||
||<#FFEBBB> '''5''' || Monday || || QA Sanity and Regressions testing ||
||<#E47A7A> '''5''' || Friday || || Verified candidate sent to device manufacturers ||
||<#E47A7A> '''6''' || Tuesday || || Results from manufacturers, Release decision made ||
||<#BB3333> '''6''' || Wednesday || || /!\ Image promotion to stable, Start of Phased Upgrades ||
||<#BB3333> '''6''' || Thursday || || End of Phased Upgrades ||


== Hotfix Releases ==

The Product Team can request a hotfix point-release at least two weeks prior to the planned release at any time between main milestones. A hotfix release is tagged with OTA-<X>.<R>, where <X> is the update that's used as base (always the latest stable OTA is used) and <R> is the release revision. The Product Team needs to provide a list of critical fixes/changes that are to be used in the hotfix release (usually as a separate LP milestone).

Hotfix releases are usually composed of critical bug and security fixes that cannot wait with their release until the next milestone. The Landing Team then prepares the snapshot PPA with the state of the latest stable release and cherry-picks packages from the stable-phone-overlay that carry the required fixes. Sometimes, if a package containing a critical fix carries too many other unrelated changes that could potentially cause regressions or significantly increase testing time, a code cherry-pick for the project can be requested. A CI Train silo is then allocated with a temporary dependency-switch to the snapshot PPA and the required package is rebuilt there with only the wanted changes.
The ubuntu-touch/rc/* channels are used for building and testing the hotfix images.

Hotfix release dates are generally very flexible but due to the nature of our QA processes usually the official start of phased upgrades is on Wednesdays.

Release Schedule for Ubuntu Touch

The LandingTeam is responsible for managing and coordinating Releases of the Ubuntu Phone image on request of the Canonical Product team.

Requirements

  • The Product Team requests an OTA update release every 6 weeks for all officially supported products
  • In case another release is required out of the standard timeframe, the Product Team needs to inform the Landing Team at least two weeks before the planned release, also officially announcing it on the mailing list (OTA point releases)
  • The Product Team hands the Landing Team a list of blocking Bugs together with the request for the milestone, this list gets published along with the above email so developers are aware of the urgency of the open issues. This bug list can be modified during the following weeks towards the milestone, the Product team is the gatekeeper for adding/removing bugs.
  • One week before the Release preparation period a translation freeze is introduced. After the freeze no landings containing changes or additions to user-visible strings will be allowed until after release.
  • During the Release preparation period (see timeline below) a snapshot of the release candidate is made to the snapshot PPA and all re-spins are prepared in the rc channel. The snapshot is updated with cherry-picks from the overlay whenever a regression needs to be fixed.
  • During the Release preparation period normal landings are enabled normally, with rc-proposed images containing the daily changes.

Timeline

Friday (US TZ) EOD (3rd week)

  • All user visible strings are now landed in the image, no silos with additional string changes are allowed to be released to the archive.

Friday (US TZ) EOD (4th week)

  • The Final Freeze gets called out by the LandingTeam. No landings are accepted by the LandingTeam or the QA Team for the selected milestone besides ship blockers and rare ProductTeam approved exceptions.

Saturday (EU TZ) (4th week)

  • The first automated candidate image is built to the rc-proposed channel.

Monday morning (EU TZ) (5th week)

  • The overlay PPA state for the candidate image is copied to the snapshot PPA.
  • The first candidate image is copied from rc-proposed to the rc channel.
  • The QA team starts the regression tests of the candidate image, in case severe regressions are found, the developer of the component gets contacted and a silo with a fix gets prepared.

  • Landings to the overlay PPA are allowed but limited by QA resources, as QA engineers prioritize release candidate testing over silos.
  • Whenever QA finds a blocking regression and a fix is prepared, the fix is cherry-picked to the snapshot PPA and a new rc image is rebuilt.

Friday EOD (US TZ) (5th week)

  • Testing of the candidate image on the desired target architectures is finished, the LandingTeam is notified about the status.

  • Information about a valid image candidate is sent to the manufacturers for testing.

Monday morning (EU TZ) (6th week)

  • The QA team runs a quick test on the community architectures that will have to be promoted alongside the product image (mako, emulator, etc) and gives a recommendation about realisability of these images to the LandingTeam.

Tuesday evening (EU TZ) (6th week)

  • The manufacturers return feedback regarding the realisability of the OTA.
  • A final decision is made based off the decisions of QA, Product management, manufacturers feedback and the Landing Team. The Product Team always have the possibility to veto any of the decisions.

Wednesday noon (EU TZ) (6th week)

  • The LandingTeam releases the Milestone. The candidate image is copied from RC to stable for all approved channels and devices (ubuntu-touch/stable/*).

  • Phased updates are started for the new stable image.

Thursday morning (EU TZ) (6th week)

  • Phased updates are over, all users should see update notifications.

Schedule Overview

Week

Day

Date for current OTA

Events

1

~Friday

Info <!> Milestone bug list announced

3

Tuesday

Warning /!\ FeatureFreeze, Warning /!\ StringFreeze

4

Friday

Warning /!\ FinalFreeze, Warning /!\ ReleaseCandidate

5

Monday

QA Sanity and Regressions testing

5

Friday

Verified candidate sent to device manufacturers

6

Tuesday

Results from manufacturers, Release decision made

6

Wednesday

Warning /!\ Image promotion to stable, Start of Phased Upgrades

6

Thursday

End of Phased Upgrades

Hotfix Releases

The Product Team can request a hotfix point-release at least two weeks prior to the planned release at any time between main milestones. A hotfix release is tagged with OTA-<X>.<R>, where <X> is the update that's used as base (always the latest stable OTA is used) and <R> is the release revision. The Product Team needs to provide a list of critical fixes/changes that are to be used in the hotfix release (usually as a separate LP milestone).

Hotfix releases are usually composed of critical bug and security fixes that cannot wait with their release until the next milestone. The Landing Team then prepares the snapshot PPA with the state of the latest stable release and cherry-picks packages from the stable-phone-overlay that carry the required fixes. Sometimes, if a package containing a critical fix carries too many other unrelated changes that could potentially cause regressions or significantly increase testing time, a code cherry-pick for the project can be requested. A CI Train silo is then allocated with a temporary dependency-switch to the snapshot PPA and the required package is rebuilt there with only the wanted changes. The ubuntu-touch/rc/* channels are used for building and testing the hotfix images.

Hotfix release dates are generally very flexible but due to the nature of our QA processes usually the official start of phased upgrades is on Wednesdays.

LandingTeam/ReleaseSchedule (last edited 2016-04-20 09:19:49 by sil2100)