LiveUsbPendrivePersistent

/Comments

Introduction

It is possible to have Ubuntu or Kubuntu on a USB drive (AKA USB Stick or Thumb drive or Flash drive) or USB hard disk drive with persistent mode. This means that you can boot from a USB drive and keep customisations such as keyboard layout, numlock, preferences, additional packages saved on the drive. This can be done using linux or windows. You will need a USB drive of 1 GB or more. This page is written after having tested the instructions on a Peak III 1 GB drive. The preparation of the drive is explained using 'fdisk' because I had errors with 'gparted' and i could not give the partitions a volume name. I used Ubuntu to make the drive. In Kubuntu it is more or less the same. Where you see 'ubuntu' replace it by 'kubuntu'. I will mark the other differences.

If you are looking for help in putting the 'alternate' cd on a pen drive, then try the instructions here: https://help.ubuntu.com/community/Installation/FromUSBStick . They're for feisty, but more or less work for gutsy too. Note that you may need to add "vesa vga=771" to the append= line in syslinux.cfg in order to get the installer to display properly.

Method 0: Automatically create Live USB system

Jaunty (9.04) onwards provide USB Startup Disk Creator system tool. It uses the usb-creator to create USB bootable startup disk on USB drive. You may also start the application from the terminal with :

gksudo usb-creator-gtk

Then choose the desired ubuntu .iso image, the usb device you want to use, erase the disk and set the degree of persistence (the slider) you need. Note that the maximum space that can be allocated for persistence is limited to 4GB (maximum file size on a FAT32 filesystem is 4GB). This limit can be overcome and is explained later on. If you do want a larger degree of persistence, set persistence to 128mb for now and continue. Hit Make Startup Disk and wait till it finishes. You are now in possession of a USB drive that can be used to run/install ubuntu on most computers. Persistence gives you the freedom to save changes, in the form of settings or files etc, during the live session and the changes are available the next time you boot via the usb drive.

To make the persistence larger (> 4GB), while still on your normal Ubuntu Installation, mount the pendrive if it is not mounted already. View the files in the FAT32 partition of the pendrive. You will now be able to see a file called casper-rw. It will be the same size as the value set on the slider. Delete this file. Open gparted :

gksudo gparted

select the live usb.

There will be a single partition only. Resize it so that you can have two partitions. You only need the current partition to be as large as it is already. You can safely move the resizing slider all the way to the left until there is just enough breathing space between the slider and the end of the first partition (coloured block). Make the newly created free space an ext4 or ext3 or ext2 partition and label it "casper-rw" without the double quotes. Hit apply changes. So now you have two partitions, the fat32 partition and the ext4 partition called casper-rw. This partition can be of any size and will be responsible for saving your data between boots. You can now boot using the usb drive and any changes you make will remain.

Portable Linux (GUI-based)

Portable Linux lets you install Ubuntu on an USB drive. Unlike Live USB creator, with this application, the free space on your USB drive is available to move files around, you get GRUB for free (so you can run MS-DOS or Memtest directly from the USB drive), and the ISO image is hidden so it's not possible to damage or delete it. You can run this application on any Linux distribution, like Fedora, not just Ubuntu.

Live USB creator (GUI-based)

WARNING: Does NOT create persistent installs on Hardy (8.04 LTS) or earlier; works only with Intrepid 8.10.

Live USB creator automates the process of creating a bootable Live USB system from a running Ubuntu Live CD. Simply run the Live CD, install the tool and start the Live USB installation from the System administration menu.

https://launchpad.net/liveusb -- probono

(A better link: https://launchpad.net/liveusb/+announcements which explains that you run the livecd and the you click on the link: http://ppa.launchpad.net/probono/ubuntu/pool/main/l/liveusb/)

{http://klik.atekon.de/liveusb/screenshot.png}

If you can't start the Live USB system from a running Ubuntu Live CD (perhaps you lack a functioning CD burner), you can actually just install liveusb to your normal system with the Ubuntu Live CD just mounted. Mount the image using the command

sudo mount -o loop /path/to/cd-image /media/cdrom0

After running the liveusb command and installing to your USB key, unmount the image with

sudo umount /media/cdrom0

Method 1: Installing Ubuntu directly to USB drive from installer CD

You can directly install Ubuntu 8.04 Hardy Heron to a USB drive using the Live CD. Simply select the USB drive as the installation target instead of the local hard disk.

Note: This will use the USB drive for /tmp, which will cause extra wear on the flash memory. If you're booting from a system with enough RAM, it would be more desirable to use a tmpfs in RAM for /tmp, in which case you'd want to copy the ISO CD image to the USB drive and add a persistent partition (see next section). On the other hand, if you're not concerned with your USB drive wearing out (lifetime warranty, wear leveling, etc), continue in this section.

Ensure in the "Installation Summary" screen that you choose to install GRUB to the USB key stick, and not the computer's hard disk (click the Advanced button to do so). Choose to install GRUB to the first USB partition (i.e. /dev/sdb1, rather than simply /dev/sdb). Failure to do this will cause the GRUB on your hard disk to be changed and render the system unbootable from the hard disk, requiring you to boot from a CD and reinstall GRUB to the hard disk (possibly requiring a chroot to the hard disk filesystem first).

You will also need to manually edit the menu.lst file of the new USB installation to change the boot references to /dev/sda, rather than /dev/sdb (or /dev/sdc etc.). This can be done by booting to the live distro mode of the Ubuntu install CD and editing the /boot/grub/menu.lst file on the USB stick. You can mount the USB stick using the Places menu -- once mounted, it can be found at /media/disk. Note: This step may not always be necessary.

Note: There is a bug with the Ubuntu 8.04 install CD that means USB key stick installs are screwed -- the desktop won't appear when you login. To get around this, you must do the following when you boot for the first time (before logging in at GDM):

1) switch to a virtual console (Ctrl+Alt+F2)

2) kill the X-server (sudo killall gdm)

3) empty the /tmp folder (sudo rm -rf /tmp/*)

4) Type startx, and perform a system update when the desktop appears (if you don't want to install a lot of software, just update the gnome-keyring package(s) -- that's the one that causes the problem)

Note that this should be fixed in the Ubuntu 8.04.1 install CD.

Method 2: hiding the ISO image in a partition in the USB drive

If you'd like to understand how the process works, the page A better way to create a customized Ubuntu Live USB drive has instructions to build a bootable, persistent USB drive with Linux.

These instructions have been tested to work correctly with Ubuntu Intrepid Ibex (8.10), and should work correctly with any distribution that uses Casper and supports persistency.

Method 3: Copying the ISO CD image to the USB drive

Preparing the USB drive

8.04 Hardy: Unfortunately, there is a bug in the versions of casper in Hardy that cause the persistent partition to not be mounted on boot. There is a workaround available, and you can download a replacement initrd.gz for 8.04.1 (place in casper directory on first partition of USB key).

Good news! It IS possible to make Feisty persistent! These instructions are needed but not sufficient. Check thishttp://www.pendrivelinux.com/2007/09/20/how-to-make-ubuntu-704-casper-persistent/. Essentially, you need to decompress initr.gz (with gzip and cpio); edit the init file adding a case called 'persistent' (see details at the link) and recreate initr.gz. I guess that without this case, the 'persistent' passed in the syslinux.cfg is simply ignored. I got persistence at the first attempt. Elfantin

Making partitions

Caution: Be careful using any disk utility to partition or format dives as a small error could damage your operating system or overwrite a whole partition of files. Remove any external drives, cameras or other storage devices connected to your computer (except for the USB drive you wish to repartition). Get familiar with the difference between 1 KB, 1 MB and 1 GB; as well as the difference between a drive (e.g. /dev/sda) and a partition (e.g. /dev/sda2). Get familiar with the drives and partitions on your computer: What are the sizes? Which is root, '/'? Which is the USB device?

Use 'disk utility' or 'gparted' to unmount the USB device, and then to delete any existing partition(s). Create a 750 MB FAT partition named 'ubuntu' (or whatever); then an ext2 partition named 'casper-rw' (exactly) on the remaining space. Select the 750 MB ubuntu partition, click "edit partition" and check "bootable". If you were successful, skip down to #Installing Ubuntu on the USB drive. If you prefer to work with 'fdisk' on the command line, keep reading below.

The preparation of the drive is explained below using 'fdisk'. You could use 'gparted' or 'disk utility' as well.

  • However I had errors with 'gparted' and I could not give the partitions a volume name. I tested 'qtparted' as well and it worked fine.

    Another issue in my case was that 'fdisk' recognised a fat32 filesystem as linux. So I checked it with gparted and it was OK.

Plug in the drive and check its device name:

mount

Assuming you have your Ubuntu set up to automatically mount your USB stick, this should show you what /media/drive or whatever it's called corresponds to on the device level.

Alternatively, you could run

sudo fdisk -l

but if you have multiple devices on your system, it will print a fair amount of fairly technical information.

We are going to suppose that its name is /dev/sdX where X is the disk device such as /dev/sda or /dev/sdc. Please be sure to substitute the correct letter for X which matches the device name you find for your pen drive. Double check it! [Note: on newer computers, which use serial ATA disks, /dev/sda is typically the main hard drive; do not overwrite this!]

Note: on some usb-sticks fdisk says "Note: sector size is 2048 (not 512)", in which case you may very well be out of luck trying to boot from it, see mailing list thread: Problem with 2GB USB Pendrive

Now we are going to create 2 partitions: 1 partition of 750 MB with a fat32 filesystem and 1 partition with the rest of the drive space with volume name "casper-rw" and ext2 filesystem. The first partition will be made active.

Open 'terminal' and unmount the USB drive (replace 'sdX1' with the appropriate partition name):

sudo umount /dev/sdX1

Fire up 'fdisk' for the drive:

sudo fdisk /dev/sdX

This will give you the fdisk prompt. Look to what you have on the drive. Check if you have to backup the data you have on it. 'p' will print the content of /dev/sdX. If you have partitions on it remove them ('d', 'partition number').

Make partition 1:

1. 'n' for new partition

2. 'p' to make it a primary partition

3. '1' to call it partition number 1

4. Then just press enter to accept the proposed starting cylinder

5. '+750M' to make its size 750 MB

6. Then 'a' to make it the active partition

Set the file system of partition 1 to FAT:

1. 't'

2. then '1'

3. then '6'

  • (I already had partition 1 selected so for me it was just 't' then '6' -Jordan_U)

    there seems to be an incongruity: when you change the filesystem to FAT using option "t" the filesystem option "6" is FAT16. w95/fat32 is option "b". Which file system needs to be used? -rebelcat1

Make partition 2:

1. 'n' for new partition

2. 'p' to make it a primary partition

3. '2' to call it partition 2

4. then just press enter to accept the proposed starting cylinder

5. then press enter to accept the proposed ending cylinder

6. Save and quit fdisk with 'w' to write the new settings.

  • I had an issue with this that every time I saved my partition setup using 'w', I'd get:

      WARNING: Re-reading the partition table failed with error22: Invalid argument.
      The kernel still uses old table.
      The new table will be available at the next reboot.
    • If this happens, and then checking with 'sudo fdisk -l' doesn't show the new partitions - here's a fix if you have a Windows OS, though I don't know why it works:

      Do nothing else with the drive. Insert into Windows (mine was XP) and format with Disk Management and use FAT32. Now, 'sudo fdisk -l' should see the two partitions under the drive. -neverard

      • I had the same error, using gparted instead of fdisk worked for me -Jordan_U

Check the result:

sudo fdisk -l

You should see the 2 partitions with the first marked with a * because it is active.

If ubuntu mounted the partitions again you should unmount them:

sudo umount /dev/sdX1
sudo umount /dev/sdX2
  • make sure (via fdisk -l) you created a w95/fat and not a linux partition, otherwise i was unable to boot from it!! (i had to use gparted to achieve this) - bernstein

    • fdisk worked for me however - booted off CD - krishnan

Alternate partition layout

Please note that Windows will only mount the first partition of any removable media - which includes USB flash drives. While you can get around this, it requires some highly kludgy driver procedures, which you are unlikely to be able to pull off on any machine where you are not an admin.

In short, having the Ubuntu Live as your first partition reduces the utility of your USB stick for what is statistically it's most likely task - transferring files to and from Windows machines.

Partition 1 : FAT32, full size of stick minus 750M
Partition 2 : FAT16, 750M  (FAT16 because older versions of syslinux don't like FAT32)

Follow the instructions, but use /dev/sdX2 instead of /dev/sdX1, and make sure you make /dev/sdX2 bootable in fdisk.

For persistence, follow the instructions to create a loopback file at LiveCDPersistence. This feature works fine from a loopback file inside a FAT32 filesystem, just as long as the file contains a supported unionfs filesystem like ext2.

Now you have (mostly) the best of both worlds.

  • You can transfer files between Windows machines on the FAT32 partition
  • You have a persistent Ubuntu thumb
  • You can mount the FAT32 partition from a persistent Ubuntu boot (manually)
  • You can enlarge or shrink your casper-rw partition without having to repartition your thumb, just make a new loopback filesystem, mount them both, and copy the files across.

If the persistent boot automatically mounted the root of the FAT32 filesystem as well as the casper-rw filesystem, it would be ideal.

Formatting the partitions

We will now format the partitions by putting a filesystem on and giving them the name 'ubuntu' (or any other name you want) and 'casper-rw' (this name is MANDATORY, do not change other names and do not use capital letters):

sudo mkfs.vfat -F 32 -n ubuntu /dev/sdX1
sudo mkfs.ext2 -b 4096 -L casper-rw /dev/sdX2

Mount the partitions on the drive. Take out the drive and put it back. Ubuntu Dapper or Edgy will mount the drive automatically if you have checked this option under System -->Preferences--> Removable media. Otherwise use 'mount':

sudo mount /dev/sdX1 /media/mountpointyouhave1
sudo mount /dev/sdX2 /media/mountpointyouhave2

On recent Ubuntu versions, the USB stick's partitions should simply appear under /media/ubuntu and /media/casper-rw if you have followed the proposed naming convention.

Installing Ubuntu on the USB drive

Download the Ubuntu image (for example, ubuntu-11.04-desktop-i386.iso) or put in an Ubuntu install CD if you have it.

If you use the downloaded image you can mount it to make it accessible as follows, using 'terminal' (supposing that the iso image is in the current directory):

mkdir ubuntuCD
sudo mount ubuntu-11.04-desktop-i386.iso ubuntuCD -o loop

You can now access the files via the 'ubuntuCD' folder.

Open the CD's top-level folder and make sure that you see the hidden files too. (verify that View-->Show Hidden Files is checked).

Copy the following folders and files to the 750 MB partition of your USB drive:

  • Copy folders: '.disk', 'casper', 'dists', 'install', 'pics', 'pool' and 'preseed' (skip the 'boot' folder);
  • Copy the folder 'isolinux' to the drive and rename it to 'syslinux';
  • Copy the file 'README.diskdefines', and optionally, the 'usb-creator.exe' and 'wubi.exe' files (skip the 'autorun.inf' and 'md5sum.txt' files);
  • Rename the file 'syslinux/isolinux.cfg' to 'syslinux/syslinux.cfg' on the USB drive by right clicking on it and selecting 'rename';
  • Rename the file 'syslinux/isolinux.bin' to 'syslinux/syslinux.bin'.

Alternately, you can run the following commands in 'terminal'. If you copy/paste, take care to make your browser window large enough to not have accidental line breaks inserted. And remember to replace /media/ubuntu with /media/whatmountpointyouhave1 if you used a different name for the mount point.

Add the -v option to cp to get some progress indication. The casper/filesystem.squashfs file in particular is large, and takes a long time to copy.

cd ubuntuCD
cp -rf casper dists install pics pool preseed .disk /media/ubuntu
cp -rf isolinux /media/ubuntu/syslinux
cd /media/ubuntu/syslinux
mv isolinux.cfg syslinux.cfg
mv isolinux.bin syslinux.bin

The FAT32 file system will not allow symbolic links to be copied over. You will get a warning about this, but it is harmless.

Unmount the iso image that you mounted above, using 'terminal'.

sudo umount ubuntuCD

Making the drive bootable

We are going to install the linux bootloader 'syslinux'. This bootloader gets its menu from 'syslinux/txt.cfg'. We will modify this file. Open 'syslinux/txt.cfg' from the USB drive with your favorite text editor. For instance:

cd /media/ubuntu/syslinux
gedit txt.cfg

The following edits need to be made:

  • Search and replace '/cdrom' with nothing;
  • Copy the first 'label' section, paste it in before itself, and make the following edits to this new section:
    • The 'label' argument should be changed to something unique, here we use 'persistent';
    • The 'menu label' should be changed to describe the option; '^Use Ubuntu in persistent mode' would seem suitable, with a caret before the "U" to make it a shortcut key for this option;
    • The word 'persistent' should be added into the 'append' line (this is the crucial point!);
  • Optionally, change the 'default' to the label name used above (e.g. 'persistent').

The following is an example of what you might end up with:

default persistent
label persistent
  menu label ^Use Ubuntu in persistent mode
  kernel /casper/vmlinuz
  append  file=/preseed/ubuntu.seed boot=casper persistent initrd=/casper/initrd.lz quiet splash --
label live
  menu label ^Try Ubuntu without installing
  kernel /casper/vmlinuz
  append  file=/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash --
label live-install
  menu label ^Install Ubuntu
  kernel /casper/vmlinuz
  append  file=/preseed/ubuntu.seed boot=casper only-ubiquity initrd=/casper/initrd.lz quiet splash --
label check
  menu label ^Check disc for defects
  kernel /casper/vmlinuz
  append  boot=casper integrity-check initrd=/casper/initrd.lz quiet splash --
label memtest
  menu label Test ^memory
  kernel /install/mt86plus
label hd
  menu label ^Boot from first hard disk
  localboot 0x80

The menu choice called 'persistent' was made the default so that when booting up you can just hit 'Enter' to start your customized session.

Install bootloader

The last thing to do is install the bootloader. Make sure you have the 'syslinux' and 'mtools' packages installed. If not, open 'synaptic' and select them from the package list to mark them to be installed, then "apply" the changes.

Or use 'aptitude' or 'apt-get' from the command line. For example, the following command should be harmless if they are already installed.

sudo aptitude install syslinux mtools

First you need to unmount your usb stick using 'disk utility', or use 'terminal' to enter the following commands (replacing 'sdX1' with the appropriate partition name for the 750 MB ubuntu partition on your USB device):

cd ~
sudo umount /dev/sdX1

This command will install the bootloader on your drive:

syslinux /dev/sdX1
  • don't know why, but i had to use the -f switch, (syslinux -f /dev/sdc1) - bernstein

    • same here - I had to use the -f switch - krishnan

    syslinux complained it couldn't create LDLINUX.SYS, but it worked in the Windows version (see below) - JanSchl├╝ter

    According to this page1 syslinux 3.35 should be used to get support for fat32, edgy ships with 3.11. According to the changelog 3.35 only seems to introduce support for subdirectories.

If you get the error:

sh: mcopy: command not found
syslinux: failed to create ldlinux.sys

then you will need to install the 'mtools' package.

Eject the drive:

eject /dev/sdX

Boot on the USB drive, change some settings, reboot again and check if the changes are kept (persistence check).

Do not forget to check if the computer you are going to use, boots USB devices first (look in BIOS settings).

So that is it. Enjoy (k)ubuntu where ever you go.

  • If you boot the drive and persistency works but when you log in to gnome you get a message saying that it couldn't write to .ICEauthority, then a simple fix is deleting .ICEauthority. You can automate that by doing this:

      echo >> .bashrc
      echo "# This makes GNOME work" >> .bashrc
      echo "rm .ICEauthority 1>/dev/null 2>/dev/null" >> .bashrc

Those using Lucid (10.04) to make a pen drive to boot Precise (12.04) that is to say, using the previous LTS release to go to the next one, will need to be aware of this incompatibility between the syslinux versions involved:

https://bugs.launchpad.net/ubuntu/+source/syslinux/+bug/608382

there are a couple of work-arounds (eg, typing 'help' at the prompt and then hitting 'enter') but the bug has been triaged and labeled WONTFIX since Maverick and as late as March 2012.

Installing Ubuntu on USB drive using Windows

UNetbootin (GUI-based, runs from either Windows or Linux)

UNetbootin automates this task by providing a GUI to create a bootable Ubuntu Live USB drive from an ISO file, and can be run from both an installed Windows or Linux system, or from a liveCD.

http://sourceforge.net/dbimage.php?id=294729

Preparing the USB drive

TODO This is not updated for Gutsy

Download a copy of the HP format utility for USB drives, this utility makes it as easy to format USB pen drives as it is to format floppies.

Don't bother wasting your time with this HP utility - it is rather useless. You can format the entire drive as FAT32 using the built in windows format (right click on the drive and select format, make sure to select FAT32 as the FS type). I found no way to partition the drive in windows, not even with PowerQuest's Partition Magic (which usually does everything you could want). Either boot off an ubuntu CD/DVD and follow the linux instructions, or if you can't do that, simply follow these instructions but use the whole flash drive for the live CD, then once you have booted from it, you can repartition and make the ext2 volume from within ubuntu itself. I simply pulled the flash drive out and re-inserted it - this ensures ubuntu is no longer using it. Not sure what the risks are - I didn't have any troubles, but perhaps you should do this at your own risk (or maybe somebody knows a safer way). If somebody knows a free utility which can partition a flash drive in winxp, please put it here!!! - mrwolf

Plug in your USB pen drive and use the HP format utility to format it with a FAT32 filesystem.

Using a partitioning tool, create two partitions. The first is formatted with a FAT32 filesystem and needs to be 750 MB. The second should occupy the remaining free space and should be formatted with the ext2 filesystem and is given the name "casper-rw". Note! Most DOS partitioning tools will create the partition name in uppercase (CASPER-RW). Since Linux is case-sensitive you need to do the formatting of the second partition in Linux. The persistent feature will not work if the partition name is in uppercase. See the Wiki for details. https://help.ubuntu.com/community/LiveCDPersistence

Installing Dapper or Edgy on the USB drive

This is the same as described in the Linux section except for accessing the downloaded image. The way to do this is to mount the iso image file and copy the files that way. You can do this with NERO or Windows VirtualCD. You might want to do it this way if you don't want to burn a CD.

Making the drive bootable

Go to http://syslinux.zytor.com/iso.php and download a copy of syslinux-3.11.zip for Windows. Extract the zip file contents and move the extracted folder to your favorite location on your hard drive.

Open WordPad and edit the file syslinux.cfg as described in the Linux section.

Open a command prompt and change directory to the location of the syslinux folder. For example, if you placed the folder on your desktop then: cd C:\Documents and Settings\(your user name)\Desktop\syslinux-3.11\win32 (by the way you DO know that command prompt has command completion just like in Linux (start entering characters and then press the TAB key to complete the entry)).

Next write the bootsectors to the flash drive by entering the following command:

syslinux -maf X:
  • where X: is replaced by the drive letter of your USB key. Be careful to pick the correct drive letter!
  • This will write the Master Boot Record (MBR), as well as mark the partition active.

See also


CategoryCleanup CategoryLive CategoryUsb CategoryBootAndPartition CategoryBootAndPartition

LiveUsbPendrivePersistent (last edited 2012-03-27 20:28:04 by nyloco)