ImageBasedUpgrades
|
Size: 2158
Comment:
|
Size: 3880
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Ubuntu is more and more interested in offering an image based system, similar to some of the other mobile operating system. Typically those images will contain a basic Ubuntu desktop system and will always be read-only (in some cases, may be signed too). |
== Introduction == With Ubuntu getting ported to more and more mobile devices, the standard update mechanism through apt and dpkg is starting to show some limitations. |
| Line 4: | Line 4: |
| User data and additional software will be stored in an overlay on top of the base image. | As a result, Ubuntu is now looking at supporting an additional way of updating devices that are based on read-only images. For those devices, we're looking at doing image based updates, basically generating new images on the server side with the usual tools (debootstrap, apt, dpkg), then generate deltas between those images and have the client pull the delta and apply it without ever actually dealing with packages. |
| Line 6: | Line 7: |
| Upgrades will typically be a simple download of a newer image and swapping the image file (likely to be squashfs). | We don't intend this to become the only way to update Ubuntu machines, and the use of apt/dpkg will remain supported with most devices having some kind of switch to opt out of image based updates and re-enable apt. |
| Line 8: | Line 9: |
| This comes with quite a few problems that are listed below and should be addresed: * Overlay filesystems typically work with copy-on-write, meaning that as soon as you install any extra package on the system, /var/lib/dpkg/status (for example) will be copied to the overlay, then then extra entry will be written to it. This is a major problem as any such system wide database will then be completely out of date after an upgrade. * Any action done by the maintainer scripts (preinst/postinst/prerm/postrm) won't be applied during the "upgrade", this will likely cause some configuration to be out of date if not completely invalid and will also cause quite a few missing entries or invalid entries in /var. * File transition won't happen as preinst won't be called, leading to more file conflicts. * In general, any file that's being touched for any reason, won't ever be upgraded. |
Blueprint: https://blueprints.launchpad.net/ubuntu/+spec/foundations-1305-image-based-updates |
| Line 14: | Line 11: |
| Some ideas on how to workaround those problems: * Write a whitelist of configuration files and database that we know how to merge, for those, run a migration script post-upgrade and for any other file detected, move them out of the way and push an error to the crash database for tracking. * Don't attempt to merge anything, instead use apt-clone before applying the upgrade to extract all the extra packages that were installed and any relevant configuration. Save those on the side, apply the upgrade and restore those. That's similar to ubiquity's "upgrade" option. * Don't allow standard package installation, instead rely on chroot or chroot-like environments that can be preserved across upgrades. |
== Goals == * Get a common "Ubuntu base system" across as many devices as possible * Update that base system through the image based system guaranteeing all devices are identical * Do all the actual package installs server-side, reducing CPU/MEM load and update time on the devices * Get a very clear split between system, apps and userdata, allowing for very simple data wipe |
| Line 19: | Line 17: |
| Feel free to update this page with more potential problems and workaround/fixes. | == Features == The basic set of feature that we want to support on all devices using image based upgrades are: * Read-only base Ubuntu system with separate read-write data storage * Secure delivery of updates made of any number of files (allowing for split device-specific/device-independent bits) * Ability to wipe all user data in one shot (factory reset) * Flexible update policy working for daily builds just as well as monthly or even yearly updates * Flexible update format, allowing for either delta based updates or full image (reformat + unpack) * Same update mechanism for any kind of devices with the only bit actually changing being the upgrader code * Per-device type procedure for factory initial flash (usually full device image) == Components == === The server === The server stores all the updates and indexes used by the client to resolve the update path for a device. Details can be found in the server wiki page linked below. === The client === The client grabs the indexes from the server, does validation on those and then finds the best update path for the device, downloads the files and pass them to the upgrader. Details can be found in the client wiki page linked below. === The upgrader === The upgrader receives a list of updates from the client, validates them and applies them to the system. Details can be found in the upgrader wiki page linked below. == Target == The current target for image based updates are Ubuntu Touch devices (phone and tablets) however the same design can easily be applied to any other device with only minimal changes to the upgrader component. The server, client and upgrader wiki pages try to be target-agnostic and so generally refer to "system partition", "user data" and "upgrader partition". Where those actually sit will vary on devices and so is detailed in device/platform specific wiki pages linked below. == Details == As this is a rather complex topic, the various pieces have been split into individual pages: * [[ImageBasedUpgrades/Server|Server implementation]] * [[ImageBasedUpgrades/Client|Client (resolver and downloader) implementation]] * [[ImageBasedUpgrades/Upgrader|Upgrader implementation]] * [[ImageBasedUpgrades/TouchDevices|Touch device setup (partitioning, mountpoints, ...)]] * [[ImageBasedUpgrades/GPG|GPG chain of trust and validation]] Additionally some extensions are still under active discussion: * [[ImageBasedUpgrades/ExtraPackageInstallation|Installation of extra packages from the archive on top of the Ubuntu base system]] |
Introduction
With Ubuntu getting ported to more and more mobile devices, the standard update mechanism through apt and dpkg is starting to show some limitations.
As a result, Ubuntu is now looking at supporting an additional way of updating devices that are based on read-only images. For those devices, we're looking at doing image based updates, basically generating new images on the server side with the usual tools (debootstrap, apt, dpkg), then generate deltas between those images and have the client pull the delta and apply it without ever actually dealing with packages.
We don't intend this to become the only way to update Ubuntu machines, and the use of apt/dpkg will remain supported with most devices having some kind of switch to opt out of image based updates and re-enable apt.
Blueprint: https://blueprints.launchpad.net/ubuntu/+spec/foundations-1305-image-based-updates
Goals
- Get a common "Ubuntu base system" across as many devices as possible
- Update that base system through the image based system guaranteeing all devices are identical
- Do all the actual package installs server-side, reducing CPU/MEM load and update time on the devices
- Get a very clear split between system, apps and userdata, allowing for very simple data wipe
Features
The basic set of feature that we want to support on all devices using image based upgrades are:
- Read-only base Ubuntu system with separate read-write data storage
- Secure delivery of updates made of any number of files (allowing for split device-specific/device-independent bits)
- Ability to wipe all user data in one shot (factory reset)
- Flexible update policy working for daily builds just as well as monthly or even yearly updates
- Flexible update format, allowing for either delta based updates or full image (reformat + unpack)
- Same update mechanism for any kind of devices with the only bit actually changing being the upgrader code
- Per-device type procedure for factory initial flash (usually full device image)
Components
The server
The server stores all the updates and indexes used by the client to resolve the update path for a device. Details can be found in the server wiki page linked below.
The client
The client grabs the indexes from the server, does validation on those and then finds the best update path for the device, downloads the files and pass them to the upgrader. Details can be found in the client wiki page linked below.
The upgrader
The upgrader receives a list of updates from the client, validates them and applies them to the system. Details can be found in the upgrader wiki page linked below.
Target
The current target for image based updates are Ubuntu Touch devices (phone and tablets) however the same design can easily be applied to any other device with only minimal changes to the upgrader component.
The server, client and upgrader wiki pages try to be target-agnostic and so generally refer to "system partition", "user data" and "upgrader partition". Where those actually sit will vary on devices and so is detailed in device/platform specific wiki pages linked below.
Details
As this is a rather complex topic, the various pieces have been split into individual pages:
Additionally some extensions are still under active discussion:
ImageBasedUpgrades (last edited 2015-05-07 13:27:29 by mail)