DVDPerformanceHacks

Summary

Currently, Ubuntu installation from a DVD takes a considerable amount of time due to the installer unnecessarily copying files for packages it will later remove, such as the many language packs that get included on the DVD image.

In order to bring the install speed more in line with the regular desktop CD, during the copy routine the installer will skip files that it knows it will later remove and that can be safely excluded from the target system without causing breakage. Packages will also be modified to use triggers where possible to further reduce unnecessary work.

Release Note

Installations from the Ubuntu desktop DVD are now significantly quicker.

Rationale

Use of DVDs as an installation medium are becoming increasingly popular, especially as the lack of space forces language packs and other useful additions off the regular desktop CD. However, the amount of time needed for an installation from DVD grows with each newly added package.

Use Cases

  • Nancy wants to quickly deploy Ubuntu from the installation DVD.

Implementation

Profiling

The first phase of the implementation will be to profile the installation routine to establish which parts are taking a long time and to determine specifically which packages take the longest to remove.

Blacklist

A blacklist for filtering out files to copy during install will be created by calling dpkg -L on the list of packages, save packages that have prerm files and are known to fail. Should this prove to be too slow, the blacklist will be constructed from /var/lib/dpkg/info/*.list. If both options prove to be inadequate, a whitelist can be constructed instead.

As some packages can fail when they're missing certain files, the list will be reviewed and throughly tested.

Triggers

It has been discovered that a number of packages, such as dictionaries and fonts, can but have not yet made use of triggers to avoid duplicate work. As part of this specification these packages will be converted to use triggers.

This list includes:

  • The aspell packages should use a trigger for update-dictcommon-aspell.
  • The myspell packages should use a trigger for update-openoffice-dicts.
  • It is possible that the TTF packages could use a trigger for defoma-font, however the defoma hint needs to stay on the filesystem but triggers happen after removal. The current proposed solution for this is to have each package write to a transient file in its prerm script that is then processed by the trigger.

APT

Calculation of removal takes a long time on the 1200+ packages. It might be possible to remove the language-support-writing packages, which would remove over 240 packages from the set. However, it is entirely possible that apt itself is slow to calculate the removal and therefore it should be profiled to see if any gains can be made.

UI Changes

Implementation of this specification will have no impact on the user interface whatsoever. It will not add any additional debconf questions.

Test/Demo Plan

The installer will be run with and without the proposed patch to determine the speed gain and identify any bugs.

BoF agenda and discussion

Installer would skip copying any files that matched a certain pattern. http://people.ubuntu.com/~ubuntu-archive/germinate-output/ubuntu.intrepid/dvd-live Some packages would fail if certain files were missing. Possible to run dpkg -L on each package to create a blacklist. ... or look at /var/lib/dpkg/info/*.list if that's too slow If a package does not have a prerm, it's safe to remove. Alternatively, we could use a whitelist.

Profiling!

  • Before doing any of this, need to confirm which parts are taking a long time
  • Calculation of removals is known to be slow
  • Instrument actual removal of individual packages to find the slowest ones

gimp-help-* has a prerm that runs install-docs -r; this appears to be safe language-pack packages appear to be safe. Might be more efficient to remove specific packages first in do_remove().

  • Discuss with mvo.

Triggerisations:

  • aspell-* should use a trigger for update-dictcommon-aspell
  • myspell-* should use a trigger for update-openoffice-dicts
  • ttf-* could possibly use a trigger for defoma-font
    • definitely need to keep defoma hint file on the file system, which means triggers are difficult since they happen after removal
    • could have each prerm write to a transient file, which is then processed by the trigger

Calculation of removal still takes a long time (1200+ packages)

  • Removing language-support-writing-* might be OK, and would remove 240+ packages
  • It is also possible that the method for calculating this removal is inefficient
    • Any inefficiencies here are likely to be in apt
    • Produce stub versions of markDelete et al that do less work, to confirm this

Comments:

  • Cd-install can also a little bit faster with some of this hacks.

CategorySpec

DVDPerformanceHacks (last edited 2008-08-06 16:59:39 by localhost)