LTSUpgrades

Differences between revisions 1 and 14 (spanning 13 versions)
⇤ ← Revision 1 as of 2007-05-14 17:18:11 →
Size: 3066
Editor: p54A67AC1
Comment:
← Revision 14 as of 2008-08-06 16:34:57 → ⇥
Size: 4726
Editor: localhost
Comment: converted to 1.6 markup
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
 * '''Launchpad Entry''': https://blueprints.launchpad.net/ubuntu/+spec/lts-upgrades  * '''Launchpad Entry''': UbuntuSpec:hardy-lts-upgrades
Line 10: Line 10:
Ubuntu does currently support version to next version release upgrades. We do not support skipping versions. We will have to change this for LTS->LTS upgrades. This spec describes what is required to implement 8.04 LTS to LTS+1 upgrades
and some of the high level problems that need to be overcome for it.

== Release Note ==

This makes it possible to do release upgrades from one long-term
support release to the next in one step.
Line 14: Line 20:
Our LTS releases are supported for 3 years on the desktop and 5 on the
server. This means that with the regular upgrades (that can't skip
versions) the user will have to perform up a lot of them (and
waste a lot of time/bandwidth).
Line 15: Line 26:

Anne is using hardy on her desktop and she does not intend to upgrade
to any other version than a LTS. When the new LTS comes out she gets a
button telling her about the new LTS release and she happily upgrades.
Line 18: Line 33:
This spec affects update-manager.
Line 20: Line 37:
== Implementation == === Python problems ===
Line 22: Line 39:
== Outstanding Issues == The changes in the python policy are challenging for the upgrader
because it is written in python. The new policy allows only packages
called "python-foo" that then contain the binary libs build against
the currently supported python versions. The python-apt package
currently contains:
{{{
/usr/lib/python2.5/site-packages/apt_pkg.so
/usr/lib/python2.4/site-packages/apt_pkg.so
}}}
Line 24: Line 49:
== BoF agenda and discussion == The old policy had packages for each python version (e.g. python2.4-apt).
This means that during the upgrade from hardy that uses python2.5
by default some binary libs may go away because the python packages
get upgraded and do no longer contain the libs for python2.5. Examples
are python-apt, python-gtk, python-vte and python-qt3.
Line 26: Line 55:
* xserver changes (dapper->edgy: xserver-xorg-video-all must be hinted to install)
  -> envy breaks, due to some broken diversions
* python/pycental transition (python2.4-foo -> python-foo)
* many file-overwrite problems and conf-file questions
* UUID change (looks good, except grub,
  check for the swap partition mkswap changes the UUID - most likely that is something that
  is not happening during a upgrade but happening during boot, most likely hibernation)
* pata->sata (mostly good, only cdroms left as problematic device)		 There are some ways around the problem, no one is very elegant:
 * provide debs/udebs with the required libs and install them into location that the upgrader is unpacked. The disadvantage is that those need to go onto the alternative CD too
 * rewrite the upgrader in C++ where propper shared libraries are available
 * copy the required python libs into the upgrader place (how to find the information?)
 * perform the upgrade from a special chroot environment and/or boot into special upgrade mode and/or provide a upgrade mode on the live-cd so that we have a stable environment while performing the upgrade
Line 35: Line 61:
* "breaks" support in dpkg, needs to be solved via the pre-requisites
* upgrader: pre-requisites support+maintainer script failure protection		 === Notification ===
Line 38: Line 63:
* xubuntu upgrade problem: theme crashes, not xubuntu specific
  -> system state changes under our feet
  -> firefox is another example		 The update-manager in hardy has a configuration file /etc/update-manager/release-upgrades that can be used to specify what upgrade options are displayed. It defaults to "lts" upgrades only and can be changed with the gnome-software-properties tool.
Line 42: Line 65:
* try to detect automatix, envy and refuse upgrade in this case (e.g. look into
  dpkg database and see if it was ever installed)
* trying as hard as possible to detect if the system was altered in some way
  that may cause breakage during the upgrade		 === Release-upgrader ===
Line 47: Line 67:
* Pre-requisites essential The release upgrader supports two different upgrade configurations.
If DistUpgrade.cfg.$fromDist is present, that will be used instead
of the regular DistUpgrade.cfg. This means that a additional
DistUpgrade.cfg.hardy needs to be created. If backports of apt/dpkg/python-apt
are required we can use prereq.hardy.list.
Line 49: Line 73:
* finding a way to make a upgrade testable (unionfs overlay, we have
  two options):
  1) test if the packages will upgrade cleanly
  2) test if the HW is working afterwards (including e.g. grub)		 === CDROM upgrades ===
Line 54: Line 75:
* there are two options for the test:
  - run the test in a virtual machine:
    - reboot into special test-mode
    - create dm COW for the upgrade tester
    - run script that will modify /etc/X11/xorg.conf to use "vesa" and
      modify /etc/fstab so that "/" is mounted from "/dev/hda" (and possibly
      other FS mount points have to change too)
    - run qemu/kvm with that COW as rootfs and run the upgrade test in it
    - when it finished, reboot again into the real system
  - run it in a unionfs
  - run it with dm-setup:
    - Create a big backing store file on the FS for a COW file
    - make it a blockdevice with loopback
    - make a special initramfs that will do the requierd setup
    - add option to grub menu "upgrade test"
    - boot into it and start up and run a upgrade
    - (future: having a option for commit the upgrade would be nifty)		 The pre-requisites for the upgrade need to go onto the CD. Build once for the previous LTS (e.g. dapper) and once for the previous regular version (e.g. gutsy). The upgrader needs to figure which one to use.
Line 73: Line 78:
== Future ==
Line 74: Line 80:
dm-snapshot documentation: http://lxr.linux.no/source/Documentation/device-mapper/snapshot.txt (also in 2.6.15)
and its avaiable in dapper!		 Finding a way to test the upgrade in a fully revertable way would
offer us a great many more testers. It should be investigated if it is
feasible to implement a test-mode. With a test-mode like this it would
be possible to find out in advance if all packages upgrade cleanly and
if not to send a problem report.

A possible way to do this is a qemu/kvm based upgrade. It would
require booting into a special test-mode where the main filesystem is
mounted ro and a device-mapper COW device is created over the real
root filesystem that is used for the upgrade test. The following steps
are required:

 * reboot into special test-mode
 * create dm COW for the upgrade tester
 * run script in COW target that will modify /etc/X11/xorg.conf to use "vesa" and modify /etc/fstab so that "/" is mounted from "/dev/hda" (and possibly other FS mount points have to change too)
 * run qemu/kvm with that COW as rootfs and run the upgrade test in it
 * when it finished, reboot again into the real system
 * copy the upgrade logs from the COW device

The dm-snapshot documentation is available at:
http://lxr.linux.no/source/Documentation/device-mapper/snapshot.txt

All the required features are available in dapper's 2.6.15 kernel.

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Summary

This spec describes what is required to implement 8.04 LTS to LTS+1 upgrades and some of the high level problems that need to be overcome for it.

Release Note

This makes it possible to do release upgrades from one long-term support release to the next in one step.

Rationale

Our LTS releases are supported for 3 years on the desktop and 5 on the server. This means that with the regular upgrades (that can't skip versions) the user will have to perform up a lot of them (and waste a lot of time/bandwidth).

Use Cases

Anne is using hardy on her desktop and she does not intend to upgrade to any other version than a LTS. When the new LTS comes out she gets a button telling her about the new LTS release and she happily upgrades.

Scope

This spec affects update-manager.

Design

Python problems

The changes in the python policy are challenging for the upgrader because it is written in python. The new policy allows only packages called "python-foo" that then contain the binary libs build against the currently supported python versions. The python-apt package currently contains: 

/usr/lib/python2.5/site-packages/apt_pkg.so
/usr/lib/python2.4/site-packages/apt_pkg.so

The old policy had packages for each python version (e.g. python2.4-apt). This means that during the upgrade from hardy that uses python2.5 by default some binary libs may go away because the python packages get upgraded and do no longer contain the libs for python2.5. Examples are python-apt, python-gtk, python-vte and python-qt3.

There are some ways around the problem, no one is very elegant:

  • provide debs/udebs with the required libs and install them into location that the upgrader is unpacked. The disadvantage is that those need to go onto the alternative CD too
  • rewrite the upgrader in C++ where propper shared libraries are available
  • copy the required python libs into the upgrader place (how to find the information?)
  • perform the upgrade from a special chroot environment and/or boot into special upgrade mode and/or provide a upgrade mode on the live-cd so that we have a stable environment while performing the upgrade

Notification

The update-manager in hardy has a configuration file /etc/update-manager/release-upgrades that can be used to specify what upgrade options are displayed. It defaults to "lts" upgrades only and can be changed with the gnome-software-properties tool.

Release-upgrader

The release upgrader supports two different upgrade configurations. If DistUpgrade.cfg.$fromDist is present, that will be used instead of the regular DistUpgrade.cfg. This means that a additional DistUpgrade.cfg.hardy needs to be created. If backports of apt/dpkg/python-apt are required we can use prereq.hardy.list.

CDROM upgrades

The pre-requisites for the upgrade need to go onto the CD. Build once for the previous LTS (e.g. dapper) and once for the previous regular version (e.g. gutsy). The upgrader needs to figure which one to use.

Future

Finding a way to test the upgrade in a fully revertable way would offer us a great many more testers. It should be investigated if it is feasible to implement a test-mode. With a test-mode like this it would be possible to find out in advance if all packages upgrade cleanly and if not to send a problem report.

A possible way to do this is a qemu/kvm based upgrade. It would require booting into a special test-mode where the main filesystem is mounted ro and a device-mapper COW device is created over the real root filesystem that is used for the upgrade test. The following steps are required:

  • reboot into special test-mode
  • create dm COW for the upgrade tester
  • run script in COW target that will modify /etc/X11/xorg.conf to use "vesa" and modify /etc/fstab so that "/" is mounted from "/dev/hda" (and possibly other FS mount points have to change too)
  • run qemu/kvm with that COW as rootfs and run the upgrade test in it
  • when it finished, reboot again into the real system
  • copy the upgrade logs from the COW device

The dm-snapshot documentation is available at: http://lxr.linux.no/source/Documentation/device-mapper/snapshot.txt

All the required features are available in dapper's 2.6.15 kernel.

CategorySpec

LTSUpgrades (last edited 2008-08-06 16:34:57 by localhost)