ImageBasedUpgrades
Differences between revisions 1 and 2
|
⇤ ← Revision 1 as of 2012-12-14 19:14:51
Size: 2158
Comment:
|
Size: 2618
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| Ubuntu is more and more interested in offering an image based system, similar to some of the other mobile operating system. Typically those images will contain a basic Ubuntu desktop system and will always be read-only (in some cases, may be signed too). |
== Introduction == With Ubuntu getting ported to more and more mobile devices, the standard update mechanism through apt and dpkg is starting to show some limitations. |
| Line 4: | Line 4: |
| User data and additional software will be stored in an overlay on top of the base image. | As a result, Ubuntu is now looking at supporting an additional way of updating devices that are based on read-only images. For those devices, we're looking at doing image based updates, basically generating new images on the server side with the usual tools (debootstrap, apt, dpkg), then generate deltas between those images and have the client pull the delta and apply it without ever actually dealing with packages. |
| Line 6: | Line 7: |
| Upgrades will typically be a simple download of a newer image and swapping the image file (likely to be squashfs). | We don't intend this to become the only way to update Ubuntu machines, and the use of apt/dpkg will remain supported with most devices having some kind of switch to opt out of image based updates and re-enable apt. |
| Line 8: | Line 9: |
| This comes with quite a few problems that are listed below and should be addresed: * Overlay filesystems typically work with copy-on-write, meaning that as soon as you install any extra package on the system, /var/lib/dpkg/status (for example) will be copied to the overlay, then then extra entry will be written to it. This is a major problem as any such system wide database will then be completely out of date after an upgrade. * Any action done by the maintainer scripts (preinst/postinst/prerm/postrm) won't be applied during the "upgrade", this will likely cause some configuration to be out of date if not completely invalid and will also cause quite a few missing entries or invalid entries in /var. * File transition won't happen as preinst won't be called, leading to more file conflicts. * In general, any file that's being touched for any reason, won't ever be upgraded. |
== Goals == * Get a common "Ubuntu base system" across as many devices as possible * Update that base system through the image based system guaranteeing all devices are identical * Do all the actual package installs server-side, reducing CPU/MEM load and update time on the devices * Get a very clear split between system, apps and userdata, allowing for very simple data wipe |
| Line 14: | Line 15: |
| Some ideas on how to workaround those problems: * Write a whitelist of configuration files and database that we know how to merge, for those, run a migration script post-upgrade and for any other file detected, move them out of the way and push an error to the crash database for tracking. * Don't attempt to merge anything, instead use apt-clone before applying the upgrade to extract all the extra packages that were installed and any relevant configuration. Save those on the side, apply the upgrade and restore those. That's similar to ubiquity's "upgrade" option. * Don't allow standard package installation, instead rely on chroot or chroot-like environments that can be preserved across upgrades. |
== Features == The basic set of feature that we want to support on all devices using image based upgrades are: * Read-only base Ubuntu system with separate read-write data storage * Secure delivery of updates made of any number of files (allowing for split device-specific/device-independent bits) * Ability to wipe all user data in one shot (factory reset) * Flexible update policy working for daily builds just as well as monthly or even yearly updates * Flexible update format, allowing for either delta based updates or full image (reformat + unpack) * Same update mechanism for any kind of devices with the only bit actually changing being the upgrader code * Per-device type procedure for factory initial flash (usually full device image) |
| Line 19: | Line 25: |
| Feel free to update this page with more potential problems and workaround/fixes. | == Target == The current target for image based updates are Ubuntu Touch devices (phone and tablets) however the design should consider potential extension to other devices. == Details == As this is a rather complex topic, the various pieces have been split into individual pages: * [[ImageBasedUpgrades/Server|Server side implementation]] * [[ImageBasedUpgrades/Client|Client side / downloader implementation]] * [[ImageBasedUpgrades/Upgrader|Upgrader side / flasher implementation]] * [[ImageBasedUpgrades/TouchDevices|Touch device setup (partitioning, mountpoints, ...)]] * [[ImageBasedUpgrades/GPG|GPG chain of trust and validation]] |
Introduction
With Ubuntu getting ported to more and more mobile devices, the standard update mechanism through apt and dpkg is starting to show some limitations.
As a result, Ubuntu is now looking at supporting an additional way of updating devices that are based on read-only images. For those devices, we're looking at doing image based updates, basically generating new images on the server side with the usual tools (debootstrap, apt, dpkg), then generate deltas between those images and have the client pull the delta and apply it without ever actually dealing with packages.
We don't intend this to become the only way to update Ubuntu machines, and the use of apt/dpkg will remain supported with most devices having some kind of switch to opt out of image based updates and re-enable apt.
Goals
- Get a common "Ubuntu base system" across as many devices as possible
- Update that base system through the image based system guaranteeing all devices are identical
- Do all the actual package installs server-side, reducing CPU/MEM load and update time on the devices
- Get a very clear split between system, apps and userdata, allowing for very simple data wipe
Features
The basic set of feature that we want to support on all devices using image based upgrades are:
- Read-only base Ubuntu system with separate read-write data storage
- Secure delivery of updates made of any number of files (allowing for split device-specific/device-independent bits)
- Ability to wipe all user data in one shot (factory reset)
- Flexible update policy working for daily builds just as well as monthly or even yearly updates
- Flexible update format, allowing for either delta based updates or full image (reformat + unpack)
- Same update mechanism for any kind of devices with the only bit actually changing being the upgrader code
- Per-device type procedure for factory initial flash (usually full device image)
Target
The current target for image based updates are Ubuntu Touch devices (phone and tablets) however the design should consider potential extension to other devices.
Details
As this is a rather complex topic, the various pieces have been split into individual pages:
ImageBasedUpgrades (last edited 2015-05-07 13:27:29 by mail)