Ubuntu Wiki

   1 [12:01] <cjohnston> Alan Bell is a director of The Open Learning Centre, a software consultancy company in the UK that works exclusively with Free and Open Source software. Alan and his family all use Ubuntu, however the chickens in the garden seem to prefer Kubuntu.
   2 [12:01] <AlanBell> hi all
   3 [12:01] <AlanBell> This session is about partitioning, and general arrangements of hard drives. Partitioning is a way of dividing up the storage on a hard drive so you can do different things with different bits of it.
   4 [12:02] <AlanBell> Feel free to ask questions as we go along, I am happy to answer anything about partitioning and also anything about chickens.
   5 [12:02] <AlanBell> and yes, my chickens really do use Kubuntu http://people.ubuntu.com/~alanbell/chicken.png
   6 [12:02] <AlanBell> Partitioning is a bit of an advanced topic and was originally scheduled at the end of the user day, but as that 03:00 UK time I figured I would probably pass out before it started so got it shifted.
   7 [12:03] <AlanBell> I don't want to start by putting off my audience, but if you are looking to install Ubuntu for the very first time this session may be overkill, just run through the standard installer and it will make sensible default choices for you.
   8 [12:03] <AlanBell> It will either create one big partition for your whole disk or if you have another operating system installed it may offer to resize it and install alongside.
   9 [12:04] <AlanBell> This session is for those who want to arrange their disks in a more specialised way, and to teach some of the history and low level background to the terminology around partitioning.
  10 [12:04] <AlanBell> So why might you want to do anything different with your partitions?
  11 [12:04] <AlanBell> The most common reason is to separate /home from the rest of the operating system.
  12 [12:05] <AlanBell> abhi_nav: I guess they are khickens or something yes!
  13 [12:05] <AlanBell> That means that with a bit of care you can totally nuke from orbit the rest of the disk and reinstall ubuntu or another distro, or any UNIX like operating system that wants to see a /home partition.
  14 [12:06] <AlanBell> You can have a dual boot system allowing you to boot into for example Ubuntu and Debian but both operating systems having the same /home partition.
  15 [12:06] <AlanBell> Just remember a load of settings live in /etc along with the user passwords (unless you are using an external directory such as LDAP to authenticate against) /home isn't quite everything that is important.
  16 [12:07] <AlanBell> It could be that you want *Extreme* performance!
  17 [12:07] <AlanBell> In theory, if you create one partition of about a quarter of the total size of the drive that should be on the outer rim of the platter and spinning faster and all the data on it closer together so the read head doesn't have to move much to get to any given bit of data. You waste 75% of the drive, but hey it might be a bit faster!
  18 [12:07] <AlanBell> If anyone actually tries that I would be interested in some benchmark results to see if it really makes any difference whatsoever!
  19 [12:08] <AlanBell> that is called short stroking. Seems a bit excessive to me.
  20 [12:08] <AlanBell> You could also have multiple file systems optimised for different use patterns, for example log files and couchdb databases only ever get appended to so you might not want a filesystem that is optimised for lots of small dynamic files for that bit of your data. To be honest I think that is over-thinking the situation again.
  21 === tech__ is now known as purvesh
  22 [12:09] <AlanBell> partitions impose a rather firm size limit on their contents, you can take advantage of this, particularly on a server by putting things that grow in separate partitions from things that get upset if the disk runs out of space
  23 [12:10] <AlanBell> so putting /var/log in a partition means that if it fills up the logging breaks, but other stuff should carry on
  24 [12:11] <AlanBell> anyone think of other reasons to use a more complex than default partition system?
  25 [12:11] <AlanBell> kbmonkey: yes, we will come on to drive letters and the numbering system in Ubuntu
  26 [12:12] <ClassBot> JR0cket76 asked: does a more advanced partitioning scheme make backups easier or harder?
  27 [12:12] <AlanBell> yay, got classbot working
  28 [12:13] <AlanBell> hmm, good question
  29 [12:13] <AlanBell> generally you back up mounted filesystems rather than partitions
  30 [12:14] <AlanBell> so I don't think the number of partitions would make that much difference from a backup perspective
  31 [12:14] <ClassBot> abhi_nav asked: I have ubuntu installed on one single partition. can i separate my /home now? with as possible as less effort?
  32 [12:15] <AlanBell> yes you can, probably with more effort than would be desired!
  33 [12:15] <AlanBell> basically you have to copy /home to another partition, then mount the other partition on top of where /home was
  34 [12:16] <ClassBot> abhi_nav asked: I am wishing for distro hopping. so do you recommend me to make my /home separate now at this point? because as I said now alrady my ubuntu is setup very nicely and it is working very good. I dotn want to ruin. it.
  35 === xfact is now known as xfact|afk
  36 [12:17] <AlanBell> yeah, not a bad idea, if you actually want to work and access your stuff on multiple distros
  37 [12:17] <AlanBell> however do watch out for stuff like different versions of firefox getting upset with the shared .mozilla area and so on.
  38 [12:18] <AlanBell> ok, lets carry on a bit
  39 [12:18] <AlanBell> Lets look at what partitions really are.
  40 [12:18] <AlanBell> A disk is a spinning hunk of magnetised metal called the platter, with a read write head floating just above the surface which can move back and forth along a radius of the circular platter.
  41 [12:18] <AlanBell> There are normally multiple platters stacked on the same axle or spindle, and multiple read/write heads. The ring of the platters with the heads at a certain position is called a cylinder
  42 [12:19] <AlanBell> so cylinders are kind of the bit of disk traced out by the heads as the disk spins, remember I mentioned putting data in the outer 25% of the disk, that is what cylinders are.
  43 [12:19] <ClassBot> sebsebseb asked: Ubuntu and other distros have Ext4 now as the default file system.  However does it really matter what file system is used  for  /  or seperate /home  or seperate /boot for example?
  44 [12:19] <AlanBell> don't think so for those ones
  45 [12:20] <AlanBell> although actually a transaction log of a journaled filesystem on a really small partition like /boot might be an unneccessary overhead
  46 [12:21] <AlanBell> kcj1993: apparently the outer portion of a disk is faster, not entirely sure I believe it.
  47 [12:22] <ClassBot> sebsebseb asked: Whats a transaction log?  Also  I think that people normally go with Ext2 or Ext3 when doing seperate /boot, rather than Ext4, because of unneeded features?
  48 [12:23] <AlanBell> ext4 is a journaled filsystem, which means it writes stuff before it commits it so there is never a half written file hanging about if the power gets cut
  49 [12:25] <AlanBell> each save is a "transaction" I am not sure that ext4 actually calls it a transaction log though, I may be getting my terminology confused with more database type journaling
  50 [12:26] <AlanBell> anyhow, the outside 512 bytes of data, before the first partition is called the Master Boot Record
  51 [12:26] <AlanBell> Inside this little nugget of data there are four 16 byte areas, into which is crammed a load of information about the 4 possible partitions.
  52 [12:26] <AlanBell> You may be thinking that four partitions is a pretty low limit, and you would be right
  53 [12:26] <AlanBell> that dates back a *long* time, most limits in the PC architecture from the past have been removed, but that one is in a pretty awkward place, there really isn't room in the master boot record for anything more sophisticated
  54 [12:27] <AlanBell> The way we get more partitions is to use the first one to put stuff in, but use one of the others (generally the second one) as a holder of the "extended partitions"
  55 [12:27] <AlanBell> So partition 1 is used as a partition, i.e. for storing your stuff in.
  56 [12:27] <AlanBell> partition 2 holds the extended partitions
  57 [12:27] <AlanBell> 3 and 4 are not used (normally)
  58 [12:27] <AlanBell> partitions 5,6,7 . . . are the extended partitions that live inside partition 2
  59 [12:28] <AlanBell> lets see what that looks like for real
  60 [12:28] <AlanBell> from an Ubuntu terminal prompt try typing
  61 [12:28] <AlanBell> sudo fdisk -l /dev/sda
  62 [12:28] <AlanBell> you should get something like this
  63 [12:28] <AlanBell> http://paste.ubuntu.com/461528/
  64 [12:29] <AlanBell> ok, so lets go through this bit by bit, why did we say "sudo fdisk -l /dev/sda"
  65 [12:29] <AlanBell> sudo allows us to run dangerous commands, stuff that can mess up other users of the computer
  66 [12:30] <AlanBell> Unix like systems are designed to support many users at once and prevent them from hurting each other (Unix came from universities where they let students use them so users are presumed to be clever, clumsy and sometimes hostile!)
  67 [12:30] <AlanBell> Messing with the partition table on a shared computer could really spoil someone's day, so we use sudo (but don't worry we are not going to break anything today)
  68 [12:30] <AlanBell> fdisk is a command for manipulating "fixed disks" and the partition tables on them
  69 [12:31] <AlanBell> -l tells it to list a partition table (today "look but don't touch" is the rule, we don't want to break anything)
  70 [12:31] <AlanBell> /dev/sda is the device that we want it to examine, /dev is where all the hardware devices live
  71 [12:32] <AlanBell> sd means it is a SCSI or SATA disk (as opposed to the old IDE disks which started with "hd")
  72 [12:32] <AlanBell> and "a" means it is the first physical disk
  73 [12:32] <AlanBell> b would be the next one and so on
  74 [12:32] <AlanBell> if you plug in a USB memory stick that will be given a letter the same way
  75 [12:33] <ClassBot> kbmonkey asked: How can I see a list of drives in my system?
  76 === tech__ is now known as purvesh_
  77 [12:35] <AlanBell> if you do fdisk -l without specifying a disk it will list all that it knows about
  78 [12:35] <AlanBell> you can see them all in /dev as well
  79 === xfact|afk is now known as xfact
  80 [12:35] <AlanBell> and in /dev/disk if you want to see the UUID of each one
  81 [12:36] <AlanBell> ok, so looking at my disk http://paste.ubuntu.com/461528/ what is it telling us?
  82 [12:36] <AlanBell> lines 4-9 are various stats about the size of the disk, this is a 320GB disk
  83 [12:36] <ClassBot> Gryllida asked: why didn't you mention that I need to defragment the hdd before setting the dual boot?
  84 [12:37] <AlanBell> err, you don't really. It is a decent time to to it I suppose
  85 [12:37] <AlanBell> you should defragment before resizing a partition, because that shuffles all the data up together so it can be shrunk
  86 [12:38] <ClassBot> abhi_nav asked: does frequent partitiong the harddrive decreases its life and/or performance? or in other word how I should not do partitioning that will damage my hdd?
  87 [12:38] <ClassBot> abhi_nav asked: what is UUID?
  88 [12:38] <AlanBell> it won't change the life or performance of the drive
  89 [12:39] <AlanBell> I guess frequent partitioning increases the probability that you will accidentally delete the wrong partition! but beyond that it should be fine.
  90 [12:40] <AlanBell> a UUID is a big long number that uniquely identifies a disk, things are moving to referring to those more so they get it right if you plug the disks in a different order
  91 [12:40] <AlanBell> so the fstab file (which I will mention a bit later) can refer to disks by their device name or UUID
  92 [12:41] <ClassBot> sebsebseb asked: Ok, but only Windows partitiosn get defragmented first in that case?   I mean Linux partitions don't tend to get defragmented if ever?
  93 [12:41] <ClassBot> abhi_nav asked: you said that it wont chante the life of drive? so in other words can you say that i can partition a hdd with any number of times with any filesystem? still not damaging it?
  94 [12:41] <ClassBot> dnielsen asked: What optimizations does Ubuntu have in place to accommidate detection and correct partitioning of SSD cards, e.g. a swap partition on an SSD is a bad idea (or so sayth the mighty Dave Jones)
  95 [12:42] <AlanBell> ok, firstly yeah linux doesn't have the defrag problems that windows does. I think that is mostly because it doesn't get so upset when things get fragmented
  96 === tech__ is now known as purvesh_
  97 [12:43] <AlanBell> yeah, partition away, don't worry about it damaging stuff physically, it is just data . . . however. . . .
  98 [12:43] <AlanBell> the next question was about swap on SSD
  99 [12:44] <AlanBell> and yes, there may be an issue on some SSD or flash drives with constant writes and reads from the same point of the disk
 100 === purvesh__ is now known as purvesh
 101 [12:45] <AlanBell> I am not aware of anything that prevents you doing that in Ubuntu (the operating system doesn't really know much about whether a drive is ssd or hdd physically)
 102 [12:45] <AlanBell> but yeah, swap on SSD might be bad for the SSD.
 103 [12:46] <AlanBell> That said, it would be a pretty damn good swap area.
 104 [12:46] <ClassBot> dnielsen asked: Specifically I would like to eventually replace all my computers HDD drives with small fast SSDs and rely on network and cloud storage, but I am worried that Ubuntu currently won't correctly setup for SSD cards ad thus will cause additional unneeded wear
 105 [12:46] <ClassBot> Marceau asked: Is scratch really necessary in Linux? Is it the same as virtual memory/paging?
 106 [12:47] <AlanBell> well I don't think modern SSDs have the problems that the early ones did, I think the drive controllers shuffle stuff about to prevent that
 107 [12:48] <AlanBell> when you install ubuntu you can specify whether you want swap or not. If you have plenty of memory you don't need it.
 108 [12:48] <AlanBell> ok, so looking at my disk http://paste.ubuntu.com/461528/ what is it telling us?
 109 [12:48] <AlanBell> 11-14 is the partition table, this is a default install of Ubuntu
 110 [12:48] <AlanBell> line 12 is the details of /dev/sda1 that is the device that corresponds to partition 1 of the physical disk /dev/sda
 111 [12:48] <AlanBell> It is a bootable partition, the little asterisk tells us that
 112 [12:48] <AlanBell> next is the start and end positions of the partition, in cylinders
 113 [12:49] <AlanBell> next bit is the ID, in this instance 83 which is just a number corresponding to the fact that this partition is going to be used for a linux filesystem
 114 [12:49] <AlanBell> as opposed to ntfs or fat or one of a couple of hundred of other filesystem possibilities
 115 [12:49] <AlanBell> line 13 is /dev/sda2, this is used for nothing other than as a placeholder for the extended partitions
 116 [12:49] <AlanBell> line 14 is /dev/sda5 which is my swap partition
 117 [12:50] <AlanBell> so that is why you end up with sda2 that seems to do nothing
 118 [12:50] <AlanBell> and sda3 and 4 which are missing
 119 [12:50] <ClassBot> There are are 10 minutes remaining in the current session.
 120 [12:50] <ClassBot> Gryllida asked: why is it '/dev/'? what does it stand for?
 121 [12:50] <ClassBot> matematikaadit asked: how much memory that must we have in the case we don't want to use swap?
 122 [12:51] <AlanBell> dev stands for devices
 123 [12:51] <AlanBell> everything in linux is treated like a file that you can read and write to
 124 [12:52] <AlanBell> so a soundcard device might live in /dev too and you can read from the microphone and write to the speakers
 125 [12:52] <AlanBell> these devices are raw disks (sda) and partitions on them (sda1)
 126 [12:52] <AlanBell> matematikaadit: I would say that anything over 1GB you can get away with no swap, but the more the merrier
 127 [12:53] <ClassBot> Gryllida asked: how do I list all devices - the /dev/ ones?
 128 [12:53] <AlanBell> ls /dev
 129 [12:53] <AlanBell> Partitions are just bits of disks where raw data can be put. they don't have much organisation in them, you can't store files on them as such. For that you need a filesystem
 130 [12:54] <AlanBell> The swap partition is a little different, it has no filesystem it is just raw disk that the kernel uses to temporarily store chunks of memory that it doesn't want right now. Swap to raw disk has less overhead than using a swap file inside a filesystem
 131 [12:54] <AlanBell> you can create a filesystem in an empty partition using the mkfs command, something like
 132 [12:54] <AlanBell> sudo mkfs -t ext4 /dev/sdax
 133 [12:54] <AlanBell> Where x is the partition number you want to TOTALLY TRASH and install a filesystem on. Please don't break your computer with this command!
 134 [12:54] <AlanBell> once you have a partition created, with a filesystem on it you can mount that in your own directory structure. For example you might mount it at the /home position if that partition was to contain all your user directories.
 135 [12:55] <AlanBell> The /etc/fstab file stores information about all your filesystems and where they should be mounted as bootup.
 136 [12:55] <AlanBell> you can also mount filesystems on the fly with the mount and unmount commands.
 137 [12:55] <ClassBot> There are are 5 minutes remaining in the current session.
 138 [12:55] <ClassBot> philinux asked: I've set mine up without extended. Just used 3 primary / /swap and /home. Any problem with this
 139 [12:56] <AlanBell> so they are sda1, sda2 and sda3 right?
 140 [12:56] <AlanBell> no great problem with that, just don't use sda4 for anything else, or you won't have anywhere to put the extended partitions if you want them
 141 [12:57] <ClassBot> philinux asked: "I've set mine up without extended. Just used 3 primary / /swap and /home. Any problem with this"
 142 [12:57] <ClassBot> Marceau asked: How do you mount a partition in a directory structure
 143 [12:57] <ClassBot> philinux asked: correct
 144 [12:57] <ClassBot> abhi_nav asked: is it recommended to install on whole single partition or have separate partitoin for /,/home,/boot,/tmp etc?
 145 [12:58] <AlanBell> Marceau: sudo mkdir /mnt/myplace
 146 [12:58] <AlanBell> Marceau: sudo mount /dev/sdax /mnt/myplace
 147 [12:58] <AlanBell> so that will mount the filesystem that lives in /dev/sdax into the /mnt/myplace area of your directory structure
 148 [12:59] <AlanBell> the place you are going to mount (the mountpoint) should exist, but does not have to be empty
 149 [12:59] <AlanBell> although you won't get to the contents of it whilst something is mounted on top of it
 150 [13:00] <AlanBell> abhi_nav: for a general use desktop PC I would go with the default "one hulking great big partition" (+swap) configuration