• Launchpad Entry: jaunty-boot-performance-kernel

• Created: RobbieWilliamson

• Contributors:

• Packages affected: grub, X, module-init-tools

Summary

It seems that a 5 second boot demonstrated at the 2008 Linux Plumber's Conference has really captured the attention of Linux users. Despite the specialized configuration demonstrated by Intel, a general purpose distribution ought to be able to boot to a useable desktop much faster then the current 30-60 seconds.

This specification focuses on the components of the kernel and support applications that are active during the startup process.

Release Note

Rationale

If a system could boot sufficiently fast, then why bother with suspend/resume or hibernate ?

Use Cases

This seems obvious to me. The faster your system boots, then the sooner it is usable. Furthermore, if one could use a fast boot to avoid suspend/resume, then one could also enjoy the highest possible power conservation setting, i.e., the system is turned off.

Assumptions

• Drop usplash and cleanup boot noise. As far as I'm concerned, usplash is only there to hide the noise generated by the kernel and foundation processes before X is started. If the pause before starting X can be reduced to a few seconds, then it hardly makes sense to start usplash (which has been the source of many bugs).
• Clean up normal kernel boot messages such that only show stopper errors are printed.

Design

• Module loading is slow compared to drivers that are compiled into the kernel image. Determine the subset of modules required for a high percentage of PC platforms and build them into the kernel. This may necessitate yet another flavor (such as -generic-fast) which does _not_ have an initramfs (or an empty initramfs). Alternatively, -generic can be optimized for the standard PC (no initramfs) and all others are required to install -server. We'll have to think about how PAE support affects the i386 case, if indeed PAE is even an option in Jaunty.
• Initramfs may be avoided in some cases where the boot essential devices are built into the kernel.
• The modprobe program is slow. None of the module dependencies are cached between invocations. The big kernel lock (BKL) is also held during the module load process, effectively stalling all other kernel activity.

• Use disk read-ahead (http://dailypackage.fedorabook.com/index.php?/archives/59-Wednesday-Why-Readahead.html) when booting from latency prone media (such as spinning disk). It may even make sense to enable read-ahead on SSD media. The intent is to read blocks from disk in the order they are stored on the media and pre-populate the disk cache so that subsequent page load requests can be satisfied immediately. As a corollary to pre-caching, mount boot file systems with the noatime and nodiratime options to avoid unnecessary inode writes.

• Ensure that device probing is performed in parallel on the various hot plug buses (PCI, UCB, Firewire).
• Stop the legal fiction of late binding in linux-restricted-modules

Implementation

Kernel Config Changes

• Reduce the 5s pause in Grub down to 1s
• Modules that should be compiled in:
  • alsa core
  • ext3
  • unix
  • ac
  • fan
  • ata
  • governors

  • Complete list is at: https://wiki.ubuntu.com/KernelTeam/2.6.28-2-generic-config

• Mine the Hardware Database to get information about common hardware use cases
• Look at Smolt for hardware info as well
• Profile the kernel boot in detail
  • Create a wiki page describing how to use the new kernel boot profiling options
• Investigate how to "quiet" down the boot sequence so splash screens are not necessary

Migration

Test/Demo Plan

Unresolved issues

UDS agenda and discussion

Intro

• 5s pause in Grub
• initramfs load and expand
• Goal: to boot in 15 seconds in a normal PC?

  • actually it is: boot a kernel+modules in <4s (Can be done in SSD

  • HD, but in the spinning ones?)

Discussion

• compiling drivers in cause their init routines to run; slows booting
• 2.6.28 has bootchart support for initial kernel boot.
• There is a size limit on kernel+initramfs that can be loaded into memory
• modprobe is really slow because it does loads of string stuff to find a module, then loading the module is non-trivial locking wise
  • pciv8086d3945sv0000sd0000magic == CARD

  • modules.aliases converts *v8086v3945* -> iwl3945

  • modules.dep to convert iwl3945 -> /lib/modules/<ver>/.../iwl3945.ko

  • insmod loads those
  • if we cache these answers we are going to save a load of time, and there are very few live answers

• Question: Why not improve the design of modprobe?

• Question: Why does the Big Kernel Lock have to be held during the whole module loading process?

Solutions

• Readahead and fast-boot kernel AKA 5 secs boot (Arjan van de Ven )
  • kernel patch to do file load auditing was rejected by upstream - so
  • no other improvement over readahead
    • It should be possible to get this functionality do with with auditd which is already in-kernel
• Detecting modules at 1st boot, then build the modules into the kernel, depending on each PC (Can be done?)
• OR detect modules at 1st boot, then cache them and load those at successive boots.
• Meta module
  • All modules required on particular HW is linked together into a meta module
• If new hardware is connected, recache the modules.
• Modules that should be compiled in:
  • alsa core
  • ext3
  • unix
  • ac
  • fan
  • ata
  • governors
  • Every single hardware driver a given computer needs

Action Items

• Get a list of modules that cane very safely be built into the kernel
• - ext, fan, etc
• OEM team experiment with metamodules, see what groupings are effective
• Mine the Hardware Database to get information about common hardware use cases
• Look at Smolt for hardware info
• Create a wiki page describing how to use the new kernel boot profiling options

--- end of first half of session ---

Readahead

• which blocks to read and in what order?
• prefetch adds auditing and accounting to page cache
• Can we record load times of f.s. blocks (in jiffies?) - very low overhead.

Action items

• Examine what's needed in the kernel to maintain a list of blocks and audit them later. This will be present in Jaunty as a seed to collect data for improving future releases
• publish a wiki page to track this get rid of l-r-m and dkms it, would speed up boot. No tempfs, no C compiler (rtg)

Next topic: Perform device probing in parallel on different busses

Next topic: "Stop the legal fiction of late binding in linux-restricted-modules"

Discussion: can we use the kernel audit hooks to save a stream that can be reassembled into a boot description with time stamps?

initramfs - at the top, see whether the root fs exists, then use it Action items:

• Scott to look at klibc and initramfs

End of session

=========================================== Foundations Team - Boot Performance ===========================================

Reference platform for boot chart is:

• - Dell Mini 9
  • Good for catching race conditions on the low end of market
  • SSD based
  • Cheap, anyone can go get one
  • Stable hardware selection, won't change within a selective timeframe
  • Intentionally choose ATOM vs core. ATOM is much better in this case.

Suggested Improvements to boot time: - udev and hal walk the tree of devices, this should only be done once - X takes 7-8 seconds to start (Desktop Team) - CUPS doesn't need to be loaded until the user needs to print for the first time. - Many of shell scripts run at every boot (major effect because they are not portable to other parts of the boot, not flexible enough). Proposed solution is to have a mountall binary that works though udev. - Setting the keyboard, fonts, can be done through udev. We currently don't do this today. When you set those things in the kernel, you do this in the first tty. The other 7 will inherent these. - usplash (creates tty8) and kills the work that was done previously.

• Plymouth as a replacement for usplash
• Starting usplash then X on tty1

-init-ng independent daemon - (easier to say apache depends bind) more modular, starts all services simultaneously.

-launchd (Apple) - inet superserver, starts and listens on ports and paths. all services are started when you first talk to them. Drawbacks on launchd: changing hardware. Proposed solution: We can do something similar though dbus, with upstart service activation.

---

CategorySpec