KarmicGrub2

Differences between revisions 1 and 10 (spanning 9 versions)
Revision 1 as of 2009-06-11 12:31:14
Size: 5222
Editor: 82-69-40-219
Comment: start
Revision 10 as of 2009-09-04 18:13:16
Size: 9004
Editor: 82-69-40-219
Comment: update OS selector section with details from implementation
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
 * '''Contributors''':  * '''Contributors''': ColinKing, DavidBarth, RobertMillan, and others
Line 12: Line 12:
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. See also CategorySpec for examples.
Line 16: Line 14:
This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.
New installations now use [[http://grub.enbug.org/|GRUB 2]] as their default boot loader. If you still need to use GRUB Legacy for some reason, then add the `grub-installer/grub2_instead_of_grub_legacy=false` option to the kernel command line at installation (press F6 at the CD boot menu) and let us know why!
Line 22: Line 18:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. The traditional version of GRUB, now known as "GRUB Legacy", is no longer supported upstream. The last release (0.97) was made in 2005, and its developers explicitly [[http://www.gnu.org/software/grub/grub-legacy.en.html|state]] that they no longer accept any new features. Unfortunately, GRUB 2 has been in a rather early state of development for many years, and when we have looked at it in the past it has not been feasible to switch to it by default. As a result, in order to support new features such as the ext4 filesystem or moving the distribution to use UUIDs everywhere, we have been compelled to do the work ourselves. Distributions have been gradually diverging in terms of which GRUB Legacy patches they include, and so they all end up with different bugs.
Line 24: Line 20:
== User stories ==

== Assumptions ==
This state of affairs finally appears to be changing. While there are still a few areas where GRUB 2 is not at feature-parity with GRUB Legacy (notably password handling and some issues around default boot options), it now appears to be in a state where the number of regressions is likely to be small enough that we can deal with them feasibly. Meanwhile, GRUB 2 offers new features such as proper EFI support (which is becoming increasingly important as some PC manufacturers begin to switch over to it), video mode support which we could use to integrate smoothly with kernel modesetting, Unicode text display, and others; there is even full support for [[http://grub.gibibit.com/|graphical menus]] under development. Given that the risks look manageable, now is a good time for us to build experience with the new features so that we can confidently deliver them when they are really needed.
Line 30: Line 24:
You can have subsections that better describe specific parts of the issue. By and large, GRUB 2 appears to be in a reasonably good state now, after a long period when we were unable to seriously contemplate a switch. There are some things to fix, of course, but for the most part these look tractable. The kernel team did extensive [[KernelTeam/Grub2Testing|testing]], which suggests a strong performance.
Line 34: Line 28:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: === General issues ===
Line 36: Line 30:
=== UI Changes === There are still some minor issues to fix; Ubuntu branding to the configuration file is not quite right.
Line 38: Line 32:
Should cover changes required to the UI, or specific UI that is required to implement this LVM: Currently GRUB cannot deal with /boot being on LVM, so we need to use LILO in that case. We will check whether GRUB 2 can deal with /boot on LVM (it definitely has code to deal with LVM, so should be a simple matter of configuration), and if so remove the need to ship LILO on our CD images.
Line 40: Line 34:
=== Code Changes === XFS: In the past, GRUB had a race condition installing on XFS. This was fixed in GRUB Legacy; ensure that it's fixed in GRUB 2 as well.
Line 42: Line 36:
Code changes should include an overview of what needs to change, and in some cases even the specific details. EFI/UEFI: We will check support on Intel-based Macs and on x86 PCs with EFI support. (Supply of hardware for the latter case is still rather limited.)

Power``PC: While it is not required for this specification to be considered implemented, it would be useful for Power``PC porters to look at the existing Open``Firmware support and see if we can switch from yaboot. This will need changes in `grub-installer` and `ubiquity`.

=== UI work ===

On systems with multiple operating systems installed, we would like to be able to offer a convenient desktop interface (e.g. via the "Restart..." menu item) to determine which OS will be started at the next boot. GRUB Legacy had `grub-reboot` and `grub-set-default` which could be used for this: `grub-reboot` made a single-shot change, while `grub-set-default` made a permanent change. While GRUB 2 does not quite have this packaged as neatly yet, the primitives are in place (`grub-editenv`, etc.) and it is expected to be simple to put them together. The upstream developers would be interested in integrating this if we beat them to it.

The graphical OS selector is defined as part of this blueprint: [[https://blueprints.edge.launchpad.net/ubuntu/+spec/dx-karmic-os-switcher|dx-karmic-os-switcher]]
The UI portion of it is explained here: [[https://wiki.ubuntu.com/DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Boot%20with%20Graphical%20OS%20Selector|DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Boot%20with%20Graphical%20OS%20Selector]]

At the engineering level, the selector is planned to interact with the system through:
 * the format of the boot entries in a grub menu.lst
 * or the format of the boot entries in the grub2 menu.cfg files (*)
 * the `grub-reboot` command

(*) Note: if both are present, the selector can determine which boot loader is installed by checking whether the `grub` package is installed, or one of the `grub-*` platform packages (excluding `grub-common`); these packages conflict so only one may be installed at once.

By default, we will hide the GRUB menu, but need to make it possible to access it when required by holding down a key. We need to make sure that the key involved is distinct from that used to instruct usplash to show detailed boot output, and also distinct from typical keys used to access firmware setup.

 * GRUB Legacy: timeout, interrupted by holding Escape
 * PC BIOSes: typical keys include Shift, Delete, and F12, but this is usually accompanied by a message such as "Press Shift to enter CMOS Setup" which disappears before the BIOS is finished, so it is not so important to avoid clashes with this
 * Apple Mac: Alt (Option) accesses built-in firmware boot menu; no visual indication of when this should be pressed
 * usplash: not implemented yet, so whatever we like

The general consensus appears to be to use Shift to access the GRUB 2 menu, and as reinforced in the UI design spec [[https://wiki.ubuntu.com/DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Bootloader|DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Bootloader]].
Line 46: Line 65:
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
(Completion of the tasks identified here is not mandatory for the completion of this specification. We have a migration process already that minimally works, and it is acceptable to consider only new installations as long as we do not actively break the upgrade case. However, improvements here are expected to reduce confusion.)

Although we do not expect to migrate all existing Ubuntu installations to GRUB 2 at this point (upgrading the boot loader being an inherently risky operation), only install it by default for new installations, we would still like to make the upgrade process as smooth as possible. It is already reasonably straightforward thanks to work done by Debian: when you install the `grub-pc` package, you get a "Chainload into GRUB 2" menu item which lets you test whether GRUB 2 will work, and an `upgrade-from-grub-legacy` command that you can run to switch the primary boot loader to GRUB 2.

A smoother approach might be to use GRUB Legacy's `grub-reboot` to arrange to boot into GRUB 2 with zero timeout, and if this works then either use `grub-set-default` to arrange to chainload GRUB 2 by default with zero timeout (cf. http://grub.enbug.org/Hiddenmenu) or run `upgrade-from-grub-legacy`. If we chainload, then GRUB Legacy's stage 1.5 and stage 2 remain integral parts of the boot process, and must not be removed.

Remember that the mapping between Linux device names and GRUB device names in `/boot/grub/device.map` may not be correct. In any migration process, it is important not to rely on this, since that might result in incorrectly overwriting boot sectors that do not already belong to GRUB.

(In the UDS session, we also discussed a problem that was believed to exist whereby `grub` would remove `/boot/grub/menu.lst` on purge. This is in fact not true; it only removes the record of that file in `ucf`, which is fine. This comment is included so that we do not need to have the discussion again!)
Line 53: Line 77:
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage. Significant testing on reasonably recent hardware has [[KernelTeam/Grub2Testing|already been performed]], but testing on older hardware will be important. Some ideas for this include:
Line 55: Line 79:
This need not be added or completed until the specification is nearing beta.

== Unresolved issues ==

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

== BoF agenda and discussion ==

{{{
Capture of meeting discussion 27/05/2009 12:00-13:00 UDS Barcelona

* Some minor issues to fix, e.g. tidy up config updates

* grub1 -> grub2 upgrade

 Go for chainloading grub2 in the upgrade
 Grub 2 for new installation
 Karmic +1 to drop grub 1?

 Make update-grub only update grub2

 Do automatic change to grub 2:

   - detect chainloaded from grub 1
   - then install grub 2
 change menu.lst in grub 1 - with zero timeout and chainload grub2

    leave stage1.5 (don't scrub it)
    - grub loads grub stage 1.5 (in 32K start of partition)
    - then loads grub stage 2 from fs
    - then chainloads grub2 (2nd stage)

    Note: do not remove grub stage1.5 + stage2

   - device map (hdX,Y) may not 100% correct on some systems

  * purge process of grub1 removes menu.list - this is an issue that needs fixing.
   - dependency issues:
     make sure post rm is removed from grub
     grub-pc replaces grub
     use breaks to stop clobbering of menu.list on purge
  
 * LVM:
    - currently grub cannot boot from /boot for LVM, needs to use lilo.
    - check if grub2 can boot /boot LVM - and hence remove the need for the lilo on ISO

 * xfs: grub2 was known to have race condition installing grub - check racy code is now fixed

 * grub2 EFI/UEFI
   Mac and x86 PC with EFI support needs checking
   need to check EFI bootloaders on x86 H/W when we get the hardware
  
* PPC OpenFirmware?

* Modifying the default boot options (e.g. default OS to boot and timeouts) once booted to change bootup next time around:

  legacy grub has grub-reboot and grub-setdefault to change boot. Can this be achieved in grub2? grub2 seems to
  provide ability to do this but had different commands to do this.

  e.g. for reboot - show menu, for failed kernels

  grub-reboot
    - for single shot mode - do once, go back to default mode thereafter
  grub-setdefault
    - e.g. for last good boot

* Boot key
  user holds key - e.g. press shift shows menu, otherwise just continue boot process
  Make sure we use different key in grub2 from that in usplash:
    grub 1 - timeout and escape
    grub 2 - shift, alt, control, (should not use alt, Apple Macs)
          make sure grub2 uses shift to get into grub2 menu
    usplash - use a different key

* Older Hardware:

 - grub2 - test on Certification Farm
 - xubuntu - ask community to test as they probably have more legacy Hardware
 - debian
 - get community to test https://wiki.ubuntu.com/KernelTeam/Grub2Testing
}}}
 - Canonical's hardware certification farm
 - Xubuntu community may well have more legacy hardware
 - close coordination with Debian (expected to move to GRUB 2 for Squeeze)
 - further promulgation of [[KernelTeam/Grub2Testing]]

Summary

Move to GRUB 2 as the default boot loader.

Release Note

New installations now use GRUB 2 as their default boot loader. If you still need to use GRUB Legacy for some reason, then add the grub-installer/grub2_instead_of_grub_legacy=false option to the kernel command line at installation (press F6 at the CD boot menu) and let us know why!

Rationale

The traditional version of GRUB, now known as "GRUB Legacy", is no longer supported upstream. The last release (0.97) was made in 2005, and its developers explicitly state that they no longer accept any new features. Unfortunately, GRUB 2 has been in a rather early state of development for many years, and when we have looked at it in the past it has not been feasible to switch to it by default. As a result, in order to support new features such as the ext4 filesystem or moving the distribution to use UUIDs everywhere, we have been compelled to do the work ourselves. Distributions have been gradually diverging in terms of which GRUB Legacy patches they include, and so they all end up with different bugs.

This state of affairs finally appears to be changing. While there are still a few areas where GRUB 2 is not at feature-parity with GRUB Legacy (notably password handling and some issues around default boot options), it now appears to be in a state where the number of regressions is likely to be small enough that we can deal with them feasibly. Meanwhile, GRUB 2 offers new features such as proper EFI support (which is becoming increasingly important as some PC manufacturers begin to switch over to it), video mode support which we could use to integrate smoothly with kernel modesetting, Unicode text display, and others; there is even full support for graphical menus under development. Given that the risks look manageable, now is a good time for us to build experience with the new features so that we can confidently deliver them when they are really needed.

Design

By and large, GRUB 2 appears to be in a reasonably good state now, after a long period when we were unable to seriously contemplate a switch. There are some things to fix, of course, but for the most part these look tractable. The kernel team did extensive testing, which suggests a strong performance.

Implementation

General issues

There are still some minor issues to fix; Ubuntu branding to the configuration file is not quite right.

LVM: Currently GRUB cannot deal with /boot being on LVM, so we need to use LILO in that case. We will check whether GRUB 2 can deal with /boot on LVM (it definitely has code to deal with LVM, so should be a simple matter of configuration), and if so remove the need to ship LILO on our CD images.

XFS: In the past, GRUB had a race condition installing on XFS. This was fixed in GRUB Legacy; ensure that it's fixed in GRUB 2 as well.

EFI/UEFI: We will check support on Intel-based Macs and on x86 PCs with EFI support. (Supply of hardware for the latter case is still rather limited.)

PowerPC: While it is not required for this specification to be considered implemented, it would be useful for PowerPC porters to look at the existing OpenFirmware support and see if we can switch from yaboot. This will need changes in grub-installer and ubiquity.

UI work

On systems with multiple operating systems installed, we would like to be able to offer a convenient desktop interface (e.g. via the "Restart..." menu item) to determine which OS will be started at the next boot. GRUB Legacy had grub-reboot and grub-set-default which could be used for this: grub-reboot made a single-shot change, while grub-set-default made a permanent change. While GRUB 2 does not quite have this packaged as neatly yet, the primitives are in place (grub-editenv, etc.) and it is expected to be simple to put them together. The upstream developers would be interested in integrating this if we beat them to it.

The graphical OS selector is defined as part of this blueprint: dx-karmic-os-switcher The UI portion of it is explained here: DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Boot%20with%20Graphical%20OS%20Selector

At the engineering level, the selector is planned to interact with the system through:

  • the format of the boot entries in a grub menu.lst
  • or the format of the boot entries in the grub2 menu.cfg files (*)
  • the grub-reboot command

(*) Note: if both are present, the selector can determine which boot loader is installed by checking whether the grub package is installed, or one of the grub-* platform packages (excluding grub-common); these packages conflict so only one may be installed at once.

By default, we will hide the GRUB menu, but need to make it possible to access it when required by holding down a key. We need to make sure that the key involved is distinct from that used to instruct usplash to show detailed boot output, and also distinct from typical keys used to access firmware setup.

  • GRUB Legacy: timeout, interrupted by holding Escape
  • PC BIOSes: typical keys include Shift, Delete, and F12, but this is usually accompanied by a message such as "Press Shift to enter CMOS Setup" which disappears before the BIOS is finished, so it is not so important to avoid clashes with this
  • Apple Mac: Alt (Option) accesses built-in firmware boot menu; no visual indication of when this should be pressed
  • usplash: not implemented yet, so whatever we like

The general consensus appears to be to use Shift to access the GRUB 2 menu, and as reinforced in the UI design spec DesktopExperienceTeam/KarmicBootExperienceDesignSpec#Bootloader.

Migration

(Completion of the tasks identified here is not mandatory for the completion of this specification. We have a migration process already that minimally works, and it is acceptable to consider only new installations as long as we do not actively break the upgrade case. However, improvements here are expected to reduce confusion.)

Although we do not expect to migrate all existing Ubuntu installations to GRUB 2 at this point (upgrading the boot loader being an inherently risky operation), only install it by default for new installations, we would still like to make the upgrade process as smooth as possible. It is already reasonably straightforward thanks to work done by Debian: when you install the grub-pc package, you get a "Chainload into GRUB 2" menu item which lets you test whether GRUB 2 will work, and an upgrade-from-grub-legacy command that you can run to switch the primary boot loader to GRUB 2.

A smoother approach might be to use GRUB Legacy's grub-reboot to arrange to boot into GRUB 2 with zero timeout, and if this works then either use grub-set-default to arrange to chainload GRUB 2 by default with zero timeout (cf. http://grub.enbug.org/Hiddenmenu) or run upgrade-from-grub-legacy. If we chainload, then GRUB Legacy's stage 1.5 and stage 2 remain integral parts of the boot process, and must not be removed.

Remember that the mapping between Linux device names and GRUB device names in /boot/grub/device.map may not be correct. In any migration process, it is important not to rely on this, since that might result in incorrectly overwriting boot sectors that do not already belong to GRUB.

(In the UDS session, we also discussed a problem that was believed to exist whereby grub would remove /boot/grub/menu.lst on purge. This is in fact not true; it only removes the record of that file in ucf, which is fine. This comment is included so that we do not need to have the discussion again!)

Test/Demo Plan

Significant testing on reasonably recent hardware has already been performed, but testing on older hardware will be important. Some ideas for this include:

  • - Canonical's hardware certification farm - Xubuntu community may well have more legacy hardware - close coordination with Debian (expected to move to GRUB 2 for Squeeze)

    - further promulgation of KernelTeam/Grub2Testing

See also

Previous (misnamed) specification in this series: FoundationsTeam/Specs/Grub2ByDefault

Kernel team testing page


CategorySpec

FoundationsTeam/Specs/KarmicGrub2 (last edited 2009-09-04 18:13:16 by 82-69-40-219)