BackupYourSystem

Differences between revisions 2 and 19 (spanning 17 versions)
Revision 2 as of 2005-07-17 23:10:49
Size: 6691
Editor: 81-179-215-74
Comment: trivial: added section title
Revision 19 as of 2006-04-21 15:42:52
Size: 10829
Editor: dsl-roigw4-feb2f800-91
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
'''Note: This page needs work. Use at your own risk. It is recommended that you read the whole page before doing anything'''
Line 33: Line 35:
tar cvpzf backup.tgz / --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys tar -cvpzf /backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /
Line 37: Line 39:
 * The 'tar' part is, obviously, the program we're going to use.
 * 'cvpfz' are the options we give to tar, like 'create archive' (obviously), 'preserve permissions'(to keep the same permissions on everything the same), and 'gzip' to keep the size down.
 * Next, the name the archive is going to get. backup.tgz in our example.
 * Next comes the root of the directory we want to backup. Since we want to backup everything; /
 * 'tar' is the program used to do a backup
 * c - create a new backup archive
 * v - verbose mode, tar will print what it's doing to the screen
 * p - preserve permissions, keeps all file permissions the same
 * z - compress the backup file with 'gzip' to make it smaller
 * f <filename> - specifies where to store the backup, /backup.tgz is the file used in this example
Line 42: Line 46:
 * After all of the options is the directory we want to backup. Since we want to backup everything we use / for the root directory

If you want to exclude all other filesystems you can use the 'l' option instead of --exclude. The above command would thus be:

{{{
tar -cvpzlf /backup.tgz --exclude=/lost+found --exclude=/backup.tgz /
}}}
Line 45: Line 56:
Well, if the command agrees with you, hit enter (or return, whatever) and sit back&relax. This might take a while. Well, if the command agrees with you, hit enter (or return, whatever) and sit back & relax. This might take a while.
Line 47: Line 58:
Afterwards you'll have a file called backup.tgz in the root of your filessytem, which is probably pretty large. Now you can burn it to DVD or move it to another machine, whatever you like! Afterwards you'll have a file called backup.tgz, which is probably pretty large, in the root of your filesytem. Now you can burn it to DVD or move it to another machine; whatever you like!

attachment:IconsPage/IconWarning3.png WARNING: Files that are bigger than 2GB (a little less actually) are not supported by ISO9660 and may or may not be restorable. So don't simply burn a DVD with a huge .iso file on it. Split it up using the command split (see man page) or use a different way to get it onto the DVD. One possiblity (untested) is the following:

{{{
sudo tar --create --bzip2 --exclude /tmp --one-file-system --sparse / | growisofs -use-the-force-luke -Z /dev/hda=/proc/self/fd/0
}}}

Note that this only backs up one file system. You might want to use --exclude instead of --one-file-system to filter out the stuff you don't want backed up. This assumes your DVD drive is /dev/hda. This will not create a mountable DVD. To restore it you will reference the device file:

{{{
sudo tar --extract --bzip2 --file /dev/hda
}}}
Line 57: Line 80:
tar cvpjf backup.tar.bz2 / --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys tar -cvpjf /backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /
}}}

=== Backup over network ===

If the filesystem is low on space and you can't mount another filesystem to store the backup file on it is
possible to use netcat to trasfer the backup.

On the receiving end you'll setup netcat to write the backup file like this:

{{{
nc -l -p 1024 > backup.tar.bz2
}}}

Then you pipe the tar command without the 'f' flag through netcat on the sending end like this:

{{{
tar -cvpj <all those other options> / | nc -q 0 <receiving host> 1024
}}}

In the above commands 1024 is just a random portnumber, anything from 1024 and up should work.

If all goes well the backup will be piped through the network without touching the filesystem being read.
With a really fast network this could actually be faster then writing the backup file back to disk.

A variation (which I just dreamed up, so I can't testify on its reliability) on the above is this command:

{{{
tar -cvpj <all those other options> / | ssh <remote host> "cat > backup.tar.bz2"
Line 64: Line 115:
Well, we'll just continue with our example from the previous chapter; the file backup.tgz in the root of the partition. Well continue with our example from the previous chapter which created the file backup.tgz in the root directory.
Line 68: Line 119:
One of the beautiful things of Linux is that This'll work even on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live-cd, but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though! One of the beautiful things of Linux is that this will even work on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live cd but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though!
Line 74: Line 125:
tar xvpfz backup.tgz -C / tar -xvpzf /backup.tgz -C /
Line 80: Line 131:
tar xvpfj backup.tar.bz2 -C / tar -xvpjf backup.tar.bz2 -C /
Line 83: Line 134:
The x option tells tar to extract the file. The -C <directory> option tells tar to change to a specific directory ( / in this example ) before extracting.
Line 86: Line 138:
Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories you excluded: Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories were excluded ( /proc, /lost+found, /mnt, /sys, etc.).
Line 89: Line 141:
mkdir proc mkdir lost+found mkdir mnt mkdir sys etc... mkdir /proc /lost+found /mnt /sys
Line 105: Line 157:
=== dd ===
The command:
{{{
dd -if /dev/hda1 > partitionimage.dd
}}}
will backup a partition. A whole drive could be backed up using just /dev/hda as the input "file". Restoring is done by:
{{{
dd -if partitionimage.dd -of /dev/hda1
}}}
You can also pipe through gzip or bzip2 to compress the images.


=== rsync ===

Rsync updates the copies the files that have changed and even then only transfers the parts of those files that have changed. That is useful for saving bandwidth when backing up over the network.
For safety, transfer between two machines is done via SSH. Rsync is especially good for backing up home directories.

The command for transfering to a remote machine is:
{{{
sudo rsync --delete -azvv -e ssh /home remoteuser@remotehost.remotedomain:./backupdirectory
}}}

{{{-z}}} compresses the data
{{{--delete}}} deletes files that don't exist on the system being backed up. Maybe you want this, maybe not.
{{{-a}}} preserves the date and times of the files (same as {{{-t}}}), descends recursively into all directories (same as {{{-r}}}), copies symlinks as symlinks (same as {{{-l}}}), preserves file permissions (same as {{{-p}}}), preserves groups (same as {{{-g}}}), preserves file ownership (same as {{{-o}}}), and preserves devices as devices (same as {{{-D}}}).
{{{-vv}}} increases the verbosity of the reporting process

===other options===
Line 108: Line 189:
 * [http://www.rsnapshot.org/ rsnapshot]
 * [http://www.csua.berkeley.edu/~emin/source_code/dibs/ DIBS]

CategoryDocumentation CategoryCleanup

See the full discussion in this thread on the Ubuntu forums. http://www.ubuntuforums.org/showthread.php?t=70566

Note: This page needs work. Use at your own risk. It is recommended that you read the whole page before doing anything

This guide to backup your system using tar to create compressed archives was taken from the post on the Ubuntu Forum written by Heliode. See the thread for discussion: http://www.ubuntuforums.org/showthread.php?t=35087

TableOfContents

Introduction

Hi, and welcome to the Heliode guide to successful backing-up and restoring of a Linux system!

Most of you have probably used Windows before you started using Ubuntu. During that time you might have needed to backup and restore your system. For Windows you would need proprietary software for which you would have to reboot your machine and boot into a special environment in which you could perform the backing-up/restoring (programs like Norton Ghost). During that time you might have wondered why it wasn't possible to just add the whole c:\ to a big zip-file. This is impossible because in Windows, there are lots of files you can't copy or overwrite while they are being used, and therefore you needed specialized software to handle this.

Well, I'm here to tell you that those things, just like rebooting, are Windows Crazy Things (tm). There's no need to use programs like Ghost to create backups of your Ubuntu system (or any Linux system, for that matter). In fact; using Ghost might be a very bad idea if you are using anything but ext2. Ext3, the default Ubuntu partition, is seen by Ghost as a damaged ext2 partition and does a very good job at screwing up your data.

Backing-up

"What should I use to backup my system then?" might you ask. Easy; the same thing you use to backup/compress everything else; TAR. Unlike Windows, Linux doesn't restrict root access to anything, so you can just throw every single file on a partition in a TAR file!

To do this, become root with

sudo su

and go to the root of your filesystem (we use this in our example, but you can go anywhere you want your backup to end up, including remote or removable drives.)

cd /

Now, below is the full command I would use to make a backup of my system:

tar -cvpzf /backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/backup.tgz --exclude=/mnt --exclude=/sys /

Now, lets explain this a little bit:

  • 'tar' is the program used to do a backup
  • c - create a new backup archive
  • v - verbose mode, tar will print what it's doing to the screen
  • p - preserve permissions, keeps all file permissions the same
  • z - compress the backup file with 'gzip' to make it smaller
  • f <filename> - specifies where to store the backup, /backup.tgz is the file used in this example

  • Now come the directories we want to exclude. We don't want to backup everything since some dirs aren't very useful to include. Also make sure you don't include the file itself, or else you'll get weird results. You might also not want to include the /mnt folder if you have other partitions mounted there or you'll end up backing those up too. Also make sure you don't have anything mounted in /media (i.e. don't have any cd's or removable media mounted). Either that or exclude /media.
  • After all of the options is the directory we want to backup. Since we want to backup everything we use / for the root directory

If you want to exclude all other filesystems you can use the 'l' option instead of --exclude. The above command would thus be:

tar -cvpzlf /backup.tgz --exclude=/lost+found --exclude=/backup.tgz /

EDIT: kvidell suggests on the forum thread that we also exclude the /dev directory. I have other evidence that says it is very unwise to do so though.

Well, if the command agrees with you, hit enter (or return, whatever) and sit back & relax. This might take a while.

Afterwards you'll have a file called backup.tgz, which is probably pretty large, in the root of your filesytem. Now you can burn it to DVD or move it to another machine; whatever you like!

attachment:IconsPage/IconWarning3.png WARNING: Files that are bigger than 2GB (a little less actually) are not supported by ISO9660 and may or may not be restorable. So don't simply burn a DVD with a huge .iso file on it. Split it up using the command split (see man page) or use a different way to get it onto the DVD. One possiblity (untested) is the following:

sudo tar --create --bzip2 --exclude /tmp --one-file-system --sparse / | growisofs -use-the-force-luke -Z /dev/hda=/proc/self/fd/0

Note that this only backs up one file system. You might want to use --exclude instead of --one-file-system to filter out the stuff you don't want backed up. This assumes your DVD drive is /dev/hda. This will not create a mountable DVD. To restore it you will reference the device file:

sudo tar --extract --bzip2 --file /dev/hda 

EDIT2: At the end of the process you might get a message along the lines of 'tar: Error exit delayed from previous errors' or something, but in most cases you can just ignore that.

Alternatively, you can use Bzip2 to compress your backup. This means higher compression but lower speed. If compression is important to you, just substitute the 'z' in the command with 'j', and give the backup the right extension. That would make the command look like this:

tar -cvpjf /backup.tar.bz2 --exclude=/proc --exclude=/lost+found --exclude=/backup.tar.bz2 --exclude=/mnt --exclude=/sys /

Backup over network

If the filesystem is low on space and you can't mount another filesystem to store the backup file on it is possible to use netcat to trasfer the backup.

On the receiving end you'll setup netcat to write the backup file like this:

nc -l -p 1024 > backup.tar.bz2

Then you pipe the tar command without the 'f' flag through netcat on the sending end like this:

tar -cvpj <all those other options> / | nc -q 0 <receiving host> 1024

In the above commands 1024 is just a random portnumber, anything from 1024 and up should work.

If all goes well the backup will be piped through the network without touching the filesystem being read. With a really fast network this could actually be faster then writing the backup file back to disk.

A variation (which I just dreamed up, so I can't testify on its reliability) on the above is this command:

tar -cvpj <all those other options> / | ssh <remote host> "cat > backup.tar.bz2"

Restoring

attachment:IconsPage/IconWarning3.png Warning: Please, for goodness sake, be careful here. If you don't understand what you are doing here you might end up overwriting stuff that is important to you, so please take care!

Well continue with our example from the previous chapter which created the file backup.tgz in the root directory.

Once again, make sure you are root and that you and the backup file are in the root of the filesystem.

One of the beautiful things of Linux is that this will even work on a running system; no need to screw around with boot-cd's or anything. Of course, if you've rendered your system unbootable you might have no choice but to use a live cd but the results are the same. You can even remove every single file of a Linux system while it is running with one command. I'm not giving you that command though!

Well, back on-topic. This is the command that I would use:

tar -xvpzf /backup.tgz -C /

Or if you used bz2;

tar -xvpjf backup.tar.bz2 -C /

The x option tells tar to extract the file. The -C <directory> option tells tar to change to a specific directory ( / in this example ) before extracting.

attachment:IconsPage/IconWarning3.png WARNING: this will overwrite every single file on your partition with the one in the archive!

Just hit enter/return/your brother/whatever and watch the fireworks. Again, this might take a while. When it is done, you have a fully restored Ubuntu system! Just make sure that, before you do anything else, you re-create the directories were excluded ( /proc, /lost+found, /mnt, /sys, etc.).

mkdir /proc /lost+found /mnt /sys

And when you reboot, everything should be the way it was when you made the backup!

GRUB restore

Now, if you want to move your system to a new harddisk or if you did something nasty to your GRUB (like, say, install Windows), You'll also need to reinstall GRUB. There are several very good howto's on how to do that here on this forum, so i'm not going to reinvent the wheel. Instead, take a look [http://www.ubuntuforums.org/showthread.php?t=24113&highlight=grub+restore here] (forum) or here: RecoveringUbuntuAfterInstallingWindows

On the forum thread, there are a couple of methods proposed. I personally recommend the second one, posted by remmelt, since that has always worked for me.

Well that's it! I hope it was helpful!

Other Methods

dd

The command:

dd -if /dev/hda1 > partitionimage.dd

will backup a partition. A whole drive could be backed up using just /dev/hda as the input "file". Restoring is done by:

dd -if partitionimage.dd -of /dev/hda1

You can also pipe through gzip or bzip2 to compress the images.

rsync

Rsync updates the copies the files that have changed and even then only transfers the parts of those files that have changed. That is useful for saving bandwidth when backing up over the network. For safety, transfer between two machines is done via SSH. Rsync is especially good for backing up home directories.

The command for transfering to a remote machine is:

sudo rsync --delete -azvv -e ssh /home remoteuser@remotehost.remotedomain:./backupdirectory 

-z compresses the data --delete deletes files that don't exist on the system being backed up. Maybe you want this, maybe not. -a preserves the date and times of the files (same as -t), descends recursively into all directories (same as -r), copies symlinks as symlinks (same as -l), preserves file permissions (same as -p), preserves groups (same as -g), preserves file ownership (same as -o), and preserves devices as devices (same as -D). -vv increases the verbosity of the reporting process

===other options===

You might also want to check out these backup programs which will help you to make automated backups of your system:

CategoryDocumentation CategoryCleanup

See the full discussion in this thread on the Ubuntu forums. http://www.ubuntuforums.org/showthread.php?t=70566

BackupYourSystem (last edited 2008-08-06 16:26:42 by localhost)