Client
Size: 1835
Comment:
|
Size: 5929
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 2: | Line 2: |
Line 3: | Line 4: |
The client tool is used to calclate the update, download the files, validate them and then set the needed flags for the upgrader to pick them up and apply. | The [[http://packages.ubuntu.com/search?keywords=system-image&searchon=names&suite=all§ion=all|system-image client tool]] is used to calculate the update, download the files, validate them, and then perform the device- and distro-specific operation to apply the update. On ubuntu-touch, this means rebooting into recovery. On snappy, this means applying the update to the "B" partition. |
Line 5: | Line 6: |
== Requirements == * Secure download of the indexes (HTTPs + GPG) * Support for everything described in the GPG spec (revocation list, multiple keyrings, ...) |
Also, on ubuntu-touch '''all''' downloads are performed by the `ubuntu-download-manager` service. On snappy, (and optionally on touch) downloads are performed by a built-in cURL based downloader. == Overview == * Secure download of the indexes (HTTPS + GPG) * Support for everything described in the [[../GPG|GPG]] spec |
Line 10: | Line 14: |
* Flexible implementation to allow different upgrader setups with minimal changes required | * Hooks to provide customized "apply-the-upgrade" operations. |
Line 13: | Line 17: |
== Nice to have == * Bandwidth limiting is a nice to have and might be tricky to implement. |
== Implementation == |
Line 16: | Line 19: |
== Implementation == The current implementation is a command line tool, however it's expected to be turned into a DBus service that the Touch UI can drive. |
There source package is divided into three (+1 for a "development" package not normally deployed) binary packages: `system-image-common` for the base system and Python libraries; `system-image-cli` for the command line tool; `system-image-dbus` for the D-Bus service. Note that on snappy, only `system-image-cli` is used since there are no D-Bus clients (e.g. `system-settings` on touch). The cli tool also provides a much more robust functionality, allowing you to do things on the cli that you can't do over D-Bus. There is a default image server available at http://system-image.ubuntu.com but it is relatively easy to deploy your own servers. |
Line 20: | Line 24: |
Whenever called the client does the following: 1. Grab http://server/channels.json and lookup the index for the current channel. If present, also grab the device GPG keyring. 1. Grab http://server/<channel>/<model>/index.json 1. Read the current version number of the device (ubuntu-build file) 1. Look for the most recent version available 1. Resolve an upgrade path to it, minimizing download size and number of reboots |
The client does the following: 1. Grab `https://server/channels.json` and lookup the index for the current channel. If present, also grab the device GPG keyring. 1. Grab `https://server/<channel>/<model>/index.json` 1. Read the current version number of the device 1. Look for the most recent version available for the specified channel on the current device 1. Calculate the candidate upgrade paths, minimizing download size, number of reboots, and other factors (including Phased Updates) 1. Score the candidate paths (lowest score wins), yielding the images to effect the upgrade |
Line 29: | Line 36: |
1. Write the list of updates for the upgrader to use 1. Reboot into the upgrader |
1. Write the commands for upgrading the device to a file 1. Apply the update (in-place "B" partition unpacking for snappy, reboot into recovery for touch) |
Line 32: | Line 39: |
Those steps don't include all of the specific GPG validation bits required to ensure the authenticity of all files. Those are detailed in the separate GPG wiki page. | Those steps don't include all of the specific GPG validation bits required to ensure the authenticity of all files. Those are detailed in the separate [[../GPG|GPG]] wiki page. Security (e.g. what to download over https/http) is outlined in the [[../Server#security|server security section]]. == DBus API == The `system-image-dbus` process exposes a D-Bus API on the System Bus which will is used by touch processes such as the [[SoftwareUpdates#Phone|u/i in the System Settings]] to query, begin, cancel, and apply a system update. This service starts via D-Bus activation and exits automatically after a configurable amount of time (i.e. it does not run forever). It maintains state such that the client can exit and get restarted to continue the update, even across reboots. The client uses the [[../Downloader|download service]] to manage all file downloads. See the [[http://manpages.ubuntu.com/manpages/vivid/man8/system-image-dbus.8.html|manpage]] for the up-to-date description of the D-Bus interface. |
Line 35: | Line 48: |
* https://code.launchpad.net/~barry/+junk/resolver | The code is now managed under Git on Launchpad: https://code.launchpad.net/ubuntu-system-image The old Bazaar repository is preserved for read-only access. == Phased Updates == It is possible on the server to ''phase'' an update, such that only a percentage of devices will receive the update. This can be used to smoke test an update in the field, even after extensive pre-publishing testing. The way this works is: * Every terminal upgrade image (i.e. highest available on a device/channel) has a percentage assigned to it, as an integer in the set [0..100] * 0% means an image has been pulled, and no device should upgrade to it * 100% means the image is available to all devices. This is the default if the image has no explicit phase percentage assigned to it * Each device calculates its own "phase", also as an integer in the set [0..100). Inputs into this random number include the channel, target image number, the device's unique ''machine id'', and a random number * If the device's phase percentage is less than or equal to the target image's percentage, then the device is "phased in" and selects the upgrade leading to this image number * If the device's phase percentage is greater than the target image's percentage, then the device is "phased out" and will not upgrade to the target image number When a device is phased out, the candidate upgrade path scoring algorithm simply discards the winning upgrade candidate and falls back to the next lowest scoring candidate path, until a phased-in upgrade is found. If no phased-in upgrade path is found, the device will not upgrade. '''Note:''' the algorithm described above is for `system-image` 3.0 or better. The `system-image` 2.5 phase algorithm is slightly different: a few different inputs are used to calculate the device's phase, and if a winning upgrade path is phased-out, no fallback is performed; the device simply won't upgrade. Also note that in all cases, `system-image-cli` allows the user to override the device's phase percentage, thus forcing the winning path to be explicitly phased in or out. Dry-run operation of `system-image-cli` can be used to see what the scores and phase percentages are without effecting an upgrade. |
Client side
Introduction
The system-image client tool is used to calculate the update, download the files, validate them, and then perform the device- and distro-specific operation to apply the update. On ubuntu-touch, this means rebooting into recovery. On snappy, this means applying the update to the "B" partition.
Also, on ubuntu-touch all downloads are performed by the ubuntu-download-manager service. On snappy, (and optionally on touch) downloads are performed by a built-in cURL based downloader.
Overview
- Secure download of the indexes (HTTPS + GPG)
Support for everything described in the GPG spec
- Resolution of the best upgrade path based on different policies (total download size, least number of reboots, ...)
- Download and validation of the files
- Hooks to provide customized "apply-the-upgrade" operations.
- Support for suspend/resume of downloads
Implementation
There source package is divided into three (+1 for a "development" package not normally deployed) binary packages: system-image-common for the base system and Python libraries; system-image-cli for the command line tool; system-image-dbus for the D-Bus service. Note that on snappy, only system-image-cli is used since there are no D-Bus clients (e.g. system-settings on touch). The cli tool also provides a much more robust functionality, allowing you to do things on the cli that you can't do over D-Bus.
There is a default image server available at http://system-image.ubuntu.com but it is relatively easy to deploy your own servers.
Step-by-step example for an update
The client does the following:
Grab https://server/channels.json and lookup the index for the current channel. If present, also grab the device GPG keyring.
Grab https://server/<channel>/<model>/index.json
- Read the current version number of the device
- Look for the most recent version available for the specified channel on the current device
- Calculate the candidate upgrade paths, minimizing download size, number of reboots, and other factors (including Phased Updates)
- Score the candidate paths (lowest score wins), yielding the images to effect the upgrade
- Download any file needed up until the next reboot
- Validate all the files
- Write them to the cache partition
- Write the commands for upgrading the device to a file
- Apply the update (in-place "B" partition unpacking for snappy, reboot into recovery for touch)
Those steps don't include all of the specific GPG validation bits required to ensure the authenticity of all files. Those are detailed in the separate GPG wiki page.
Security (e.g. what to download over https/http) is outlined in the server security section.
DBus API
The system-image-dbus process exposes a D-Bus API on the System Bus which will is used by touch processes such as the u/i in the System Settings to query, begin, cancel, and apply a system update. This service starts via D-Bus activation and exits automatically after a configurable amount of time (i.e. it does not run forever). It maintains state such that the client can exit and get restarted to continue the update, even across reboots. The client uses the download service to manage all file downloads. See the manpage for the up-to-date description of the D-Bus interface.
Code
The code is now managed under Git on Launchpad: https://code.launchpad.net/ubuntu-system-image
The old Bazaar repository is preserved for read-only access.
Phased Updates
It is possible on the server to phase an update, such that only a percentage of devices will receive the update. This can be used to smoke test an update in the field, even after extensive pre-publishing testing. The way this works is:
- Every terminal upgrade image (i.e. highest available on a device/channel) has a percentage assigned to it, as an integer in the set [0..100]
- 0% means an image has been pulled, and no device should upgrade to it
- 100% means the image is available to all devices. This is the default if the image has no explicit phase percentage assigned to it
Each device calculates its own "phase", also as an integer in the set [0..100). Inputs into this random number include the channel, target image number, the device's unique machine id, and a random number
- If the device's phase percentage is less than or equal to the target image's percentage, then the device is "phased in" and selects the upgrade leading to this image number
- If the device's phase percentage is greater than the target image's percentage, then the device is "phased out" and will not upgrade to the target image number
When a device is phased out, the candidate upgrade path scoring algorithm simply discards the winning upgrade candidate and falls back to the next lowest scoring candidate path, until a phased-in upgrade is found. If no phased-in upgrade path is found, the device will not upgrade.
Note: the algorithm described above is for system-image 3.0 or better. The system-image 2.5 phase algorithm is slightly different: a few different inputs are used to calculate the device's phase, and if a winning upgrade path is phased-out, no fallback is performed; the device simply won't upgrade.
Also note that in all cases, system-image-cli allows the user to override the device's phase percentage, thus forcing the winning path to be explicitly phased in or out. Dry-run operation of system-image-cli can be used to see what the scores and phase percentages are without effecting an upgrade.
ImageBasedUpgrades/Client (last edited 2016-10-31 14:09:03 by localhost)