Introduction
Each release of Ubuntu Core follows a cadence of 2 weeks. During these 2 weeks the following products are tested for the release:
- Snapd SRUs in supported Ubuntu Desktop releases
- Core snap
- Core images
This document describes the different steps that new deliverables of Ubuntu Core need to take in order to be released. It includes the different gates and the validation criteria for its transition between the different channels and pockets involved in the pipeline.
The main goal is to get saner Ubuntu Core builds and be able to release it often and in a predictable date.
Reference platforms
The following reference platforms are used to validate a release:
- Kvm i386
- Kvm amd64
- Dragonboard 410c
- Raspberry Pi2
- Raspberry Pi3
Snapd SRU:
- All the supported stable Ubuntu Desktop releases.
Other distro:
- N/A
Channels definition
Edge: xenial archive + edge PPA (https://launchpad.net/~snappy-dev/+archive/ubuntu/edge)
The edge PPA receives a new snapd deb daily, it is used for builds of the core, ubuntu-core and kernel snap with automated submission to the edge channel in the store.
The edge PPA contains experimental changes and test code. It is the developer playground where cross-package changes land and plumbing development happens.
Core snap in the edge channel and packages in the edge PPA are not intended to be tested as part of the release process.
Beta: xenial archive + image PPA (https://launchpad.net/~snappy-dev/+archive/ubuntu/image)
Packages are built from the release branch into this PPA. This PPA is used to build beta and candidate images.
Candidate: xenial-archive + image PPA
Every 2 weeks, after the successful validation of the core snap in the beta channel, the core snap is promoted from the beta channel to the candidate channel (technically the candidate channel point to the new revision of the core snap)
Stable
The core snap is promoted to the stable channel after successful termination of the verification process.
Pre-Release testing tasks
Drivers:
- Ubuntu Core team
The Ubuntu Core team tries to build quality into the product by testing the new features as soon as possible and not as a separate task. The same is true for other aspects involved in the product lifecycle, like security or operations. This is done using the spread testing framework with each PR and merge in github. All the team members are involved in writing tests and keeping the build green. There are also additional testing scenarios that are automatically triggered, like specially crafted reexec conditions. This is done for amd64 and i386 architectures.
Assuming master is green and all the different scenarios are passing, the release manager prepares builds of snapd in the image PPA and a core snap in the beta channel. At the same time snapd is uploaded to the -proposed pocket of all the supported stable Ubuntu releases.
The core snap in the beta channel is verified by the Core Team using the Spread testing framework.
The verification of the core snap triggers its manual publication to the candidate channel and mark the beginning of the testing period.
Snapd SRUs
Drivers:
- Ubuntu Core release manager
- Ubuntu QA team
- Ubuntu SRU team
Target:
- Snapd from -proposed pocket in all supported Ubuntu Desktop stable releases (Default Ubuntu Desktop installation)
The release of snapd to stable releases follows a special case described in https://wiki.ubuntu.com/StableReleaseUpdates#Snapd and the QA process is described in https://wiki.ubuntu.com/SnapdUpdates. The verification is performed by the Ubuntu QA team.
Snapd is uploaded to the -proposed pocket of the target stable release at the same time than the new core snap is uploaded to the beta channel. This triggers the execution of autopkgtest which runs the integration tests (spread) shipping with the package. The results are available on the excuse page of each release (eg http://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#snapd)
The manual verification process starts when the new core snap with the same version of snapd passed testing in the beta channel is uploaded to the candidate channel.
When the verification is successful the tracking bug and any bug filed in launchpad and referenced in the changelog are marked as verification-done.
The final release to -updates is coordinated with the release of the core snap and happens on the same day.
Core snap
Drivers:
- Ubuntu Core team
- Ubuntu QA team
The core snap is verified in the channels beta then candidate using spread tests and a set of manual tests to cover non-automated tests.
Beta Channel
Drivers:
- Ubuntu Core team
Target:
- core snap from store / beta channel and all the reference platforms
The core snap in the beta channel is built from the release branch by the Ubuntu Core release manager. The verification is performed by the Ubuntu Core Team.
The core snap is verified on all the reference platform by:
- Doing the initial setup manually with console-conf
- Executing the spread tests with an image built using the new core from beta and kernel from stable. The tests are triggered manually.
- Executing the spread tests with the latest stable image after refreshing core to beta. The tests are triggered manually.
When the verification is successful the core snap is promoted to the candidate channel
Candidate Channel
Drivers:
- Ubuntu Core release manager
- Ubuntu QA team
- Ubuntu SRU team
Target:
- core snap from store / candidate channel and all the reference platforms
The core snap in the candidate channel is promoted from the beta channel by the Ubuntu Core release manager when the beta channel passes verification.
The verification is performed by the the Ubuntu QA team and Ubuntu Core team.
The core snap in the candidate channel is verified on all the reference platform by:
- Doing the initial setup manually with console-conf
- Executing the spread tests. The tests are triggered manually.
Executing the manual test plan of console conf in PractiTest. See tests description at report_console-conf_2017_04_12.pdf
Executing the upgrade tests manually in PractiTest
- Performing exploratory testing.
When the verification is successful:
- the Ubuntu Core release manager is notified and the core snap can be promoted to the stable channel on the day of the release.
- The Ubuntu Foundations team is notified to produce core images with this version of the core snap in the candidate/pending channel on cdimage.ubuntu.com.
Core images
Candidate
Drivers:
- Ubuntu Core release manager
- Ubuntu QA team
- Ubuntu Foundations team
Target:
Candidate images in http://cdimage.ubuntu.com/ubuntu-core/16/candidate/pending
The Ubuntu Foundations team produces core image builds after the successful verification of the core snap in the candidate channel.
The Ubuntu Foundations team is notified by the Ubuntu Core release manager.
The candidate images are verified on all the reference platforms by
- Doing the initial setup manually with console-conf
- Executing the spread tests. The tests are triggered manually.
After a successful verification of the images in pending/ the Ubuntu Foundations team is notified by the Ubuntu Core release manager and the images are copied to current/
Stable
Drivers:
- Ubuntu Core release manager
- Ubuntu QA team
- Ubuntu Foundations team
Target:
Stable images in http://cdimage.ubuntu.com/ubuntu-core/16/stable/pending
The Ubuntu Foundations team produces core image builds after the promotion of the core snap to the stable channel. The Ubuntu Foundations team is notified by the Ubuntu Core release manager.
This images have the same content than the images in candidate but are a rebuild not a copy and must be re-verified to ensure they are still valid.
The candidate images are verified on all the reference platforms by
- Doing the initial setup manually with console-conf
- Performing a quick sanity check of the image.
After a successful verification, the Ubuntu Foundations team is notified by the Ubuntu Core release manager and the images are copied from pending/ to current/