• Launchpad Entry: usb-installation-images

• Created:

• Contributors: EvanDandrea

• Packages affected: partman, ubiquity, grub, grub-installer

Summary

An application will be developed to convert and write Ubuntu CD images to USB disks. In addition, our existing tools will be modified to better handle installing from a removable disk.

Release Note

It is now possible to install Ubuntu from a USB disk. Just install and run the usb-installer package, then insert a USB disk into your computer to create an installation image.

Rationale

• CDs are slow, unreliable media.

Use Cases

• Evan would like to install Ubuntu, but his laptop does not have a CD-ROM drive.
• Joe would like to test and install Ubuntu, but he finds CDs to be too slow.
• Allen wants to download and try Ubuntu, but he doesn't have a CD burner to write the ISO to a disk.
• Bob wants to use Ubuntu and non-default packages, but he doesn't have internet connection at home.

Design

A utility will be written that takes a CD or ISO file as input and writes its contents in a somewhat modified form to an inserted USB disk of the user's choosing. This is done instead of shipping another binary blob because a large amount of bandwidth is already required to host cdimage, and because even if more bandwidth was available, a user friendly tool to easily write disk images to devices is not readily available on most systems anyway.

The utility will first be written for Ubuntu and then written for Windows and Mac OSX, sharing code between the three applications if possible. Only the completion of the Ubuntu utility will be required for this specification to be considered complete for Ubuntu 8.10. The Windows and OSX applications will only be considered if there is sufficient time available.

+----------------------------------------------------+
|         Create an Ubuntu desktop USB disk          |
|                                                    |
| This utility will copy Ubuntu from the inserted CD |
| onto your USB disk so that you may boot from your  |
| USB disk and try or install Ubuntu.                |
|                                                    |
| Files will be copied from:                         |
| Ubuntu 8.04 Desktop CD                             |
|                                                    |
| Select an USB disk to use:                         |
| [ SanDisk Cruzer Micro |v|]                        |
|                                                    |
| How much space should be used?  Additional space   |
| will be used for persistent storage.  If no        |
| additional space is selected, persistent storage   |
| will be disabled and changes made when in Ubuntu   |
| will not be permanent.                             |
| [=======================================V--------] |
|                1.5 GB (500 MB free)                |
|                                                    |
|                              [ Quit ] [ Install ]  |
+----------------------------------------------------+

+----------------------------------------------------+
|         Create an Ubuntu desktop USB disk          |
|                                                    |
| This utility will copy Ubuntu from the inserted CD |
| onto your USB disk so that you may boot from your  |
| USB disk and try or install Ubuntu.                |
|                                                    |
| Files will be copied from:                         |
| Ubuntu 8.04 Desktop CD                             |
|                                                    |
| Select an USB disk to use:                         |
| [ SanDisk Cruzer Micro |v|]                        |
|                                                    |
| There is insufficient space on the USB disk you    |
| have selected.  Please delete some unneeded files  |
| off the disk before continuing.                    |
|                                                    |
|                              [ Quit ] [ Install ]  | <- greyed out
+----------------------------------------------------+

+----------------------------------------------------+
|         Create an Ubuntu desktop USB disk          |
|                                                    |
| This utility will copy Ubuntu from the inserted CD |
| onto your USB disk so that you may boot from your  |
| USB disk and try or install Ubuntu.                |
|                                                    |
| Files will be copied from:                         |
| Ubuntu 8.04 Desktop CD                             |
|                                                    |
| Select an USB disk to use:                         |
| [ SanDisk Cruzer Micro |v|]                        |
|                                                    |
| How much space should be used?  Additional space   |
| will be used for persistent storage.  If no        |
| additional space is selected, persistent storage   |
| will be disabled and changes made when in Ubuntu   |
| will not be permanent.                             |
| [=======================================V--------] |
|                1.5 GB (500 MB free)                |
|                                                    |
| This USB disk already has a boot sector.  If you   |
| continue with the install it will be overwritten.  |
|                                                    |
|                              [ Quit ] [ Install ]  |
+----------------------------------------------------+

+----------------------------------------------------+
| +------------------------------------------------+ |
| |=================                               | |
| +------------------------------------------------+ |
|                  Copying files...                  |
+----------------------------------------------------+

• Like ubiquity, it will not include a cancel button once the process has started. This can later be changed, if desired, by first backing up the MBR before making any changes and recording a list of the files it copies so that canceling the process reverts any changes. If the disk was blank to start with, then we would just wipe it.
• Perisistence will be enabled by default using the existing support for a casper-sn.ext2 loopback ext2 filesystem.
• The slider for space to use will stop at 700 MB, indicating no persistance.

Support in the installer

Because, unlike CDs, USB disks are treated like regular disks, modifications will need to be made to partman, ubiquity, and grub in order to better support installation from these devices.

Implementation

Language and toolkit selection

We will use Python for the programming language as it's cross-platform and the developers are familiar with it. As it's not readily available on Mac or Windows, we can either bundle it, or in the case of Windows use IronPython to run it on the .NET CLR.

We will use GTK for the Ubuntu frontend. The Windows and Mac frontend toolkits will be determined later.

Installer changes

• Partman will need to filter out a disk if the installation files reside on it.
• As the position of a USB disk in the device list and its presence can change often, grub will need to be modified to support the use of UUIDs as a replacement for its existing device notation. Grub-installer and update-grub will also need to be modified to use UUIDs instead of the existing device notation.
• Given the above change, ubiquity will be modified to default to installing the bootloader to the UUID of the device selected during partitioning instead of its existing default of "(hd0)".

UI Changes

While modifications will need to be made to ubiquity, no user interface or string changes should be necessary.

Test/Demo Plan

• As KVM supports booting from USB devices, it will be used to test for bugs related to installing from such a device.

BoF agenda and discussion

Targets

• Ubuntu desktop installable USB key
• Netbook/Classmate installation
• Ubuntu Mobile? (much less user interaction required/desired)
• Ubuntu alternate installable USB key
• Ubuntu server installable USB key

Design

• Don't want to ship (and mirror) yet another 700M blob
• Prefer to have a tool which accepts an Ubuntu .iso as input and writes the necessary bits to a USB stick
• Linux-only tool in first iteration, Windows later (unless volunteers materialise)

  • Write in Python, use IronPython to ship .NET object code?

  • Windows can also boot a live CD and transform to USB stick

Advantages

• No CD waste
• Possibility to have 1 Gb ubuntu image (300 more Mb of space)
• No need for optical drive
• Faster. (USB has 480 Mb per second and CD-ROM 52X has 62 Mb per second)

Notes from previous discussion (UDS-Seville)

Live/Install from USB key

• Much faster for ISO testing
• Avoid "CD blanks"
• Works on machines without optical drive
• USB key is a mass storage device (a hard drive)
  • Rather than a floppy disk

Use cases

• DD'able netboot image (small, just works)

  • netboot/boot.img.gz can be written to a partition; check whether it can also just be written to the disk device and if so update docs

  • We do currently have a PXE/ddable image but it needs one too many layers to extract it

  • https://help.ubuntu.com/7.04/installation-guide/i386/boot-usb-files.html#usb-copy-easy

• DD image with 1GB partition for copying in an alternate .iso
  • installer checks for /*.iso images on all local volumes

  • https://help.ubuntu.com/7.04/installation-guide/i386/boot-usb-files.html#usb-copy-flexible

  • https://help.ubuntu.com/7.04/installation-guide/i386/boot-usb-files.html#usb-add-iso

• Live CD image set up for persistency out of the box

  • http://www.debuntu.org/how-to-install-ubuntu-linux-on-usb-bar

Results

• Modify installer to check for:

  • /ubuntu*.iso containing either a .deb local repo

    • Already done for hd-media installer (alternate-style)
  • Live Ubuntu image containing a squashfs
• Prefer copying entire file tree to USB image
  • Simpler
  • Guaranteed to work even with 4.3GB DVD images (VFAT limit is 4.0GB)

Needs research

• Relationship to smart-boot-manager
• Syslinux/isolinux
  • Syslinux wrapper writes a bootsector "blob" to the front of the CD
    • opens as vfat with mtools
    • writes out ldlinux.sys
    • copies syslinux code (ldlinux.bss) into bootsector
• We can do the equivalent under MS Windows (rawrite) provided that we ship the two files above on the CD
  • Question: can we access the raw device without Administrator privileges?

• http://www.weethet.nl/english/hardware_bootfromusbstick.php

  • Method involving the MS Windows formatting "[x] Create start up disk".

Targets

• Desktop CD/DVD on USB --- (direct dd > /dev/sdb); should work but believed to be buggy at the moment

• Desktop CD/DVD on USB with local ISO --- not interesting (except for having a small bootstrap image)
• Desktop CD/DVD on USB with persistence

  • https://wiki.ubuntu.com/LiveUsbPendrivePersistent

• Netboot on USB with network fetch
• Netboot on USB with local alternate .iso

Wants

Transformation utility that takes a .iso image and it works out what to do with it

Should work for Linux and Windows, so possibly two separate applications

• "Write to USB key?"
• MAKE THE DESKTOPCD ON USB WORK.
  • Makes ISO testing easier
  • The place where we have less technical users.
  • Persistence?
  • ALREADY COVERED BY WUBI!
• Casper to look for .iso and look inside for squashfs, mount boot that
  • Hunt though top-level directories
    • Modify checkdev in casper, recurse one level.
      • Already done for wubi installer.
• Make persistency not check for partition label
• Copy /contents/ of .iso image to vfat.
• Copy .iso to USB stick
• Fish vmlinuz and initrd out of .iso
• Run syslinux
• Make a Unix script to do this
• Ask somebody to reimplement for MS Windows
• Point to this from the CD download pages.
• Provide a mini DesktopCD dd-able image of the above.
• Make it possible to create bootable-USB-sticks for Windows 7
• Install grub-bootloader on USB-stick if multiple operating systems are selected for a single usb-device

Comments

• Providing a little windows executable to load the iso inside a qemu virtual machine would be very cool. This guy apparently did something like that. If done, the user should be warned that accessing the Ubuntu usb installation as a virtual machine will lead to much slower performance. The pro here is being able to access your ubuntu programs quickly without rebooting, yet still being able to do use the same usb key as a native ubuntu installation when needed - TacOne

• Some of us don't have CD burners in our computers, especially at work. It would be nice to be able to accomplish all of the above without having to write the ISO to a disk first. I added an extra Use Case to cover this.

  • I think you misunderstand. The specification does not say it will write the ISO to a CD, it says it will extract the contents to a USB disk. No CD drive is required for that. -- EvanDandrea

• Note that UNetbootin (Launchpad) already does much of the tasks that have been described above. It already runs from Ubuntu (and other Linux distributions), Windows, and it can be ported to Mac OSX with minimal effort. I could easily prepare a customized version that implements the features which aren't already available in the existing version, such as persistent overlay creation, and adds in Ubuntu branding. Additionally, Python-GTK is a rather poor choice for usage on Windows; it will lead to an extremely large executable size since the Python runtime and GTK libraries, which amount to several megabytes even once compressed, will have to be bundled in. I believe that C++ and Qt4 (which UNetbootin is written in) would be a much better choice for this particular task, since Qt4 has better integration with the native Windows look and feel, and there will be no issues with Qt licensing because Ubuntu is licensed under the GPL. -- GezaKovacs (UNetbootin Developer)

  • For an example of how a stripped-down, customized UNetbootin build can look and be made, see the customized builds for Ubuntu Eee at http://www.ubuntu-eee.com/ and EeePCLinuxOS at http://www.eeepclinuxos.com/ Note that though it lacks liveUSB persistence support (since it isn't supported by EeePCLinuxOS) this can easily be implemented for a potential Ubuntu version. --GezaKovacs

• Fedora has implemented this as https://fedorahosted.org/liveusb-creator . It's worth looking over their code and seeing if anything can be shared or reused. -- EvanDandrea

  • Note that liveusb-creator, like UNetbootin, is written using the Qt4 toolkit, therefore it'll share the same disadvantages of needing external libraries and not integrating into Ubuntu like a GTK application would (which appeared to be the main concern with UNetbootin). As a further disadvantage, since liveusb-creator is written in Python, it'll additionally require the Python runtime to be bundled in for the Windows version, leading to an excessively large executable file size which will hamper redistribution efforts. --GezaKovacs

• About the partition size selections. There are supposed to be two partitions on a persistent mode USB disk, where the first should be made of read-only data extracted from the CD/ISO image, and the second should hold persistent writable information, right? If that is the case, the read-only partition would have the very same ammount of stored data from creation to deletion. Would it be possible to automaticallyfigure the minimum space required, leaving maxim space for persistent storage? -- LucasHartmann

• Please see https://launchpad.net/liveusb - it does pretty much exactly what you describe -- probono

  • from looking at that page, it seems like liveusb has to be run from a live CD, so you'll need to burn the CD and boot from it first then? -- TormodVolden

---

CategorySpec