AutomaticUpgrade

Summary

This is a proposal for an application capable of easing the transition from one version of Ubuntu to the next.

Rationale

At the moment people who want to upgrade from one version of Ubuntu to the new release have to update their sources.list manually, and then do a dist-upgrade. For most people not experienced with Ubuntu, using the command-line is a big step (see CommandLineDisintegration), especially where it concerns something as important as an upgrade. Aside from this, many people may not know that a new version of Ubuntu is available.

The Starter Guide in Ubuntu 5.10 contains instructions on how to upgrade ... to Ubuntu 5.10. Instructions on how to upgrade to Ubuntu 6.04ish could not be included because they depended on knowing the codename for that release, and the codename was not announced in time.

One of the difficulties of actually doing an upgrade is being confronted with unexpected questions about how to deal with config files during the upgrade itself. These come at a bad time, since the user can't use the machine that is being upgraded to search the web for help. So it would also help if the system could simulate a "dry run" of the upgrade or look for changed configuration packages or somehow provide upgrade information tailored to the set of packages currently installed.

Use cases

Fred is running Breezy. He is using a fast Internet connection. When Dapper is released, his Update Manager notices that there's a new release available and that his system will have to be rebooted. Fred accepts the upgrade at the end of his work day, goes home, and his machine is running Dapper when he comes back next morning.

Jane does not have an Internet connection, but she got a new Dapper CD-ROM from Shipit. She puts the disk in her drive, and the same thing happens.

Joe has also installed Breezy by CD-ROM. He does have a low-bandwidth Internet connection. When Dapper is released, his system notices and asks him whether he wants to update. He declines, because his Internet connection is too slow. After a month, he too gets a new Dapper CD-ROM and upgrades his system just like Jane did.

  • s/Dapper/Dapper+1/ s/Breezy/Dapper/ ?

Scope

We are concerned with helping the user update from one release of a distribution to the next one, via either the network or a CD/DVD-ROM with .deb files.

Any other update, such as from Debian to Ubuntu, from $release to $release+2, or via the live / UbuntuExpress CD (which does not have .deb files), is out of scope.

Design

There are three steps that needs to be taken:

  • 1) The user must be informed about the new available distro
    • (possibly release notes as well) and run the upgrade tool
    2) The upgrade tool must be able to download updated information
    • how to perform the upgrade (e.g. additional steps like upgrading certain libs first)
    3) The upgrade tools runs and installs/remove packages and does
    • some additional steps like post-release cleanup

Inform the user

The user is informed about new releases with the meta-release file that is downloaded from the net. It contains information like codename, version number and date of the release. The release file fetching is part of the update-manager application. When a new distro release is found it adds some button to the gui that allows a dist-upgrade.

Get the upgrade information

A upgrade may involve more steps than upgrading the packages. It may be required to upgrade certain packages first (like libgtk2.0-0 for the hoary->breezy upgrade because of a bug that made gtk apps crash) or do other stuff that may involve changing files or other scary stuff.

Because we do not know the exact steps at the time the tools are written this upgrade information must be totally flexible. It also must not be part of the regular packaging system because the user may not have run "apt-get update" for a long time and therefore apt does not know about the existence of the package.

The information must not be in a package included in the distribution because we want to be able to solve new problems as they are discovered.

A mechanism is proposed that involves downloading a gpg signed tar.gz from a URL. That tar.gz will contain a executable that is run and may contain additional stuff that may be required (additional meta-informaton, glade-files, pixmaps etc). If any packages needs to be upgraded first (like libgtk2.0-0 for the hoary->breezy upgrade), they should be put into the "$distro-updates" component first and the upgrader can pull them from this repository. The executable "is" the upgrade information. It will do all the needed steps automatically that are usually written in the release notes.

The update tool downloads the tarball, unpacks it, runs the script and makes use of possible additional stuff inside the tarball. Any actions that are taken need to communicate with a frontend that displays a nice and user-friendly screen.

The mechanism needs to be flexible enough to support cd upgrades as well without any network sources. If the network is available, it should still check there for a new version of the upgrade tarball (embarrassing bugs may still happen...)

The actual upgrade

Before the actual upgrade, some sanity checks needs to be performed. For instance, checking that the cache has a BrokenCount == 0 or that no unusual amount of 3rd party packages is installed.

The steps that need to be taken in order to guarantee a upgrade are run next. That includes things like installing/upgrading packages before the actual dist-upgrade, modify sources.list (with a /etc/apt/sources.list.d/ entry) or run arbitrary scripts.

Because a upgrade always involves removing packages there must be some sanity checks to ensure that nothing is removed that shouldn't be. Checking if e.g. no Kernel, X, or meta-packages are removed needs to be checked. For meta-packages a additional flag (Metapackage) should be used in the Packages file. Additionally it should be possible to have a whitelist in the upgrade-package that knows about safe removals. For distributions that do not have a Metapackage flag yet (like breezy) we ship with additional meta-information about the meta-packages available for this distribution and its dependencies.

We should remove packages that are obsolete and were available in the old release only (or at least display a list of those packages). To do this, we should have a list available of obsolete packages that were available in previous releases of Ubuntu (to compare against the currently available packages) and their version numbers. In addition we may add information about packages that are replaced with something different in new version for the distribution and offer the user to automatically convert (if that wasn't done with conflicts/replace already).

The upgrade tool should be usable without a GUI: it should fall back to displaying its progress information in a terminal. People should be able to get information about the new distro even when they don't run X. These goals (provide a server upgrade) may not make it into the first version and that will probably be superseded by smart because it contains a "abstract" gui already.

Implementation

Step 1, Inform the user about the upgrade

A meta-release file with information about the available distributions is stored on a central server and it is If-Modified-Since tested on each run of update-manager. If a new version is found, update-manager shows that visually.

Step 2, getting the upgrade information

We should publish it through the launchpad archive infrastructure and use the ubuntu-archive key for it. The tarball should be called dist-upgrade.tar.gz and should be in dist/$distro release context and it should contain a executable with the same name. It is uncompressed into a temp directory. See the CustomUploadFormats spec also.

Any other arch-specific logic can be handled by the script calling dpkg --print-architecture.

Daniel Silverstone needs a Python function which installs the tarball into the dists/ tree, adds any relevant symlinks, etc. The entry for the tarball in the changes file needs to be in section "raw-dist-upgrade". We should also add upload validation (another Python callable which checks "is this a tarball, does it install all of its files in the right place, does it include an upgrade script, ...").

Step 3, install the upgrade

The tarball that was downloaded should contain a program to be invoked by UpdateManager, written in python-gtk/python-apt (with a nice abstraction for the frontend), that is able to perform dist-upgrades.

It should try to check for space constraints (as well as possible) before performing the upgrade.

We also need to protect the user's package selection (ubuntu-desktop). To do this those packages need to be protected from being removed by the package problem resolver. We expect the best results by porting aptitude's new shiny resolver to libapt and make it available through python-apt, as the new problemresolver gives the client program a lot more control over its behavior. If that can't be done in time we can use Protect() by pkgProblemResolver.

To find cruft we have to generate a list of obsolete packages:

  • 1) Unknown origin (dpkg -i, whatever): we don't know a lot about them, so in general we don't want to remove them. 2) Obsolete (in new release): those should be marked for removal automatically.

    3) Obsolete obsolete (eg obsoleted in warty -> breezy upgrades): we can do this by pre-calculating what obsoletes we had in the previous upgrades and put it in the upgrade script. Those should be marked for removal as well.

If anything from universe is installed warn the user that the upgrade may not be as smooth as it could be. Do the same for packages from foreign repositories and search the sources.list for them. Tell the user about foreign repositories and offer to edit the sources.list and tell what packages are affected.

Flag all old kernels except for the two most recent and/or the currently running one as automatically installed? (NB: this assumes integration of the auto-remove patch). Without the auto-remove patch we may as well just remove the old kernels.

GUI

The GUI should be fairly simple. As much as possible should be done automatically.

Step 1 (informing the user) will be done in update-manager. When a new release is available, an additional item will appear in the list of available upgrades, corresponding to the new release. It will contain an additional button when a new release is available. Pressing that button starts the dist-upgrade or displays the release notes. This step should be available via the command-line as well, so it should be implemented as an external python script ("dist-upgrade-bootstrap") that can be run in both gtk and cli mode.

Step 2 (downloading the information) will be done by the "dist-upgrade-bootstrap" script. It will contain a gtk fetcher dialog that downloads the complete upgrade script.

Step 3 (installing/performing additional actions) will be a python-gtk/python-apt application that basically displays three steps: 1) pre-release work 2) downloading the upgraded packages 3) installing the packages 4) doing post-install cleanup 5) present a "you should reboot message". The installation runs in maximized mode and we should warn (in some friendly way somewhere) that the desktop may misbehave during the upgrade and that restarting applications might help.

Questions about removing cruft should be asked during Step 4 (Postinstall Cleanup).

Code

Step 1 (informing the user) is done. For steps 2+3 no code has been written yet.

Data preservation and migration

Any global tasks needed to ensure that data is preserved and migrated will be handled by the upgrade tool (these may be unknown until a later point in the release cycle). Where individual packages can fulfil their own needs in this area, they should take responsibility for doing so.

Outstanding issues

No code has been written yet.

See also

See also the UDU discussion (SystemUpgradeTool)

Implementation Notes

  • the meta-release file on the server may contain a key ReleaseNotes: URI that points to a textfile (maybe html later) with the release notes for the given distro. it will be displayed before the actual upgrade

CategorySpec

AutomaticUpgrade (last edited 2008-08-06 16:19:16 by localhost)