Ubuntu Wiki

   1 [16:56] <smoser> TeTeT will talk about "UEC Persistency", which offers a way to get EBS root-like function on UEC, and even LTS 10.04.
   2 [16:56] <smoser> i wont take his thunder, though.
   3 [16:56] <ClassBot> There are 5 minutes remaining in the current session.
   4 === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud Days - Current Session: UEC persistency - Instructors: tetet
   5 [17:01] <ClassBot> Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/24/%23ubuntu-classroom.html following the conclusion of the session.
   6 [17:01] <TeTeT> Hello! Nice to have you in class today.
   7 [17:02] <TeTeT> It's a bit weird, but I'm as nervous as before giving a live class :)
   8 [17:02] <TeTeT> If you have any questions, ask them in #ubuntu-classroom-chat with the prefix
   9 [17:02] <TeTeT> QUESTION
  10 [17:02] <TeTeT> I am not very familiar with the classbot, but I'll do my best to check if there are any questions in the queue.
  11 [17:02] <TeTeT> A brief introduction: I'm Torsten Spindler, been working for Canonical since December 2006 and I am part of the corporate services team, so unlike most other presenters here, I'm on the commercial side.
  12 [17:03] <TeTeT> I bring this up as one of my responsibilities is to maintain and deliver the Ubuntu Enterprise Cloud classes.
  13 [17:03] <TeTeT> This is directly related to this session, as I present one of the case study exercises found in the UEC class, it's the latest addition to the course material and brand new
  14 [17:03] <TeTeT> So what do I want to present here?
  15 [17:04] <TeTeT> During the UEC class I often got the question: 'How can I have an instance on UEC that stores all of its information on an EBS volume, so I can simply use it like a regular virtualized server?'
  16 [17:04] <TeTeT> Why is that of interest? The data on an instance is volatile, e.g. if the instance dies, all data of it is gone.
  17 [17:04] <TeTeT> Unless you store the data on a persistent cloud datastore; in UEC we have two of them: Walrus S3 and EBS volumes.
  18 [17:05] <TeTeT> An EBS volume is a device in an instance that serves as a disk, very much like a USB stick you insert in your system.
  19 [17:05] <TeTeT> you can see a graphic for this at http://people.canonical.com/~tspindler/UEC/attach-volume.png
  20 [17:05] <TeTeT> keep in mind that the disk actually attaches to an instance running on the node controller, not on the node controller itself
  21 [17:06] <TeTeT> the technology used in UEC is 'ATA over Ethernet - AoE', which means that the EBS volume is exported via network from a server, the EBS storage controller.
  22 [17:06] <TeTeT> With Amazon Web Services (AWS) it is possible to boot from an EBS volume in the public cloud, with a technology named 'EBS root'. For more info on this, please see http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?Concepts_BootFromEBS.html
  23 [17:06] <TeTeT> With UEC for the private cloud we use Eucalyptus as upstream technology. With Eucalyptus the concept of booting an instance straight from an EBS volume is not given.
  24 [17:07] <TeTeT> So the situation looks a bit like this: http://people.canonical.com/~tspindler/UEC/01-instance-volume.png
  25 [17:07] <TeTeT> forgive my lack of design skills ;)
  26 [17:07] <TeTeT> in UEC people usually have this: http://people.canonical.com/~tspindler/UEC/02-instance-volume-standard.png
  27 [17:08] <TeTeT> an instance holds the kernel and /, while the data is saved on a persistent EBS volume
  28 [17:08] <TeTeT> UEC persistency is about moving the kernel and / to the EBS volume
  29 [17:08] <TeTeT> depicted in http://people.canonical.com/~tspindler/UEC/03-instance-volume-ebs-based-instance.png
  30 [17:08] <TeTeT> that is, the kernel of the instance launches a kernel stored on an EBS volume
  31 [17:09] <TeTeT> the kernel on the EBS volume will use / from the ebs volume and run completely from there
  32 [17:09] <TeTeT> I asked the Ubuntu server team on advice for realizing such a service back in January 2011. It was motivated by the questions I received during the UEC class.
  33 [17:10] <TeTeT> My initial thought was to have a specialized initrd that calls pivot_root on a root filesystem stored in an EBS volume.
  34 [17:10] <TeTeT> But Scott Moser (smoser) had a much better idea: Why not use kexec to load the kernel and root from the EBS volume.
  35 [17:10] <TeTeT> More information on kexec-loader can be found at http://www.solemnwarning.net/kexec-loader/
  36 [17:10] <TeTeT> Scott went ahead and implemented this and my colleague Tom Ellis tested it.
  37 [17:10] <TeTeT> Then I used the docs from Scott, tested them and created an exercise for the UEC class out of it.
  38 [17:11] <TeTeT> We decided to publish this work in Launchpad and you can find the result in https://launchpad.net/uec-persistency.
  39 [17:11] <TeTeT> The branch under lp:uec-persistency contains the needed code and the exercise in the docs directory.
  40 [17:12] <TeTeT> smoser just told me that we use something like kexec-loader, not exactly the same tech
  41 [17:12] <TeTeT> see the chat channel for more background info ;)
  42 [17:12] <TeTeT> If you don't have bazaar installed right now, you can also take a look at the exercise PDF, found at http://people.canonical.com/~tspindler/UEC/ebs-based-instance.pdf
  43 [17:12] <TeTeT> the odt file is also freely available
  44 [17:12] <TeTeT> I will now cover this exercise step by step.
  45 [17:12] <TeTeT> No questions so far on what the aim is?
  46 [17:13] <TeTeT> to repeat, we want to use the kernel and root filesystem found on an EBS volume, not that in the instance itself
  47 [17:13] <TeTeT> The steps during the exercise have to be conducted from two different system, one I named the [cloud control host], the other the [utility instance].
  48 [17:13] <TeTeT> it would be useful to have the PDF or ODT open for the exercise now
  49 [17:14] <TeTeT> With cloud control host I mean any system that holds your UEC credentials, so you can issue commands to the front end.
  50 [17:14] <TeTeT> The utility instance is created during the exercise and is used to populate an EBS volume with an Ubuntu image.
  51 [17:15] <TeTeT> The first two steps are preparing your cloud with a loader emi. This loader emi will later be used to kexec the kernel on your EBS volume.
  52 [17:15] <TeTeT> Step three and four setup the utility instance. This is a regular UEC instance that is large enough to hold an Ubuntu image and store it on the attached EBS volume.
  53 [17:15] <TeTeT> The EBS volume created and attached in step three will be the base for your EBS based instances later on.
  54 [17:16] <TeTeT> All the steps five to nine are needed to copy an Ubuntu image to the EBS volume and ensure it boots fine later on.
  55 [17:16] <TeTeT> In step 11 a snapshot of the pristine Ubuntu EBS volume is made. While one could use the EBS volume right away, it's much nicer to clone it via the snapshot mechanism of EBS.
  56 [17:17] <TeTeT> Just in case you later want another server based on that image.
  57 [17:17] <TeTeT> Steps 12 and 13 are there to launch an instance based on an EBS volume.
  58 [17:17] <TeTeT> The final step 14 describes how to ssh to the instance and check if it is really based on the EBS volume, e.g. /dev/sdb1.
  59 [17:18] <TeTeT> That's really all there is to it, thanks to smosers work. Perfectly doable by yourself within 2 - 4 hours, starting with two bare metal servers
  60 [17:18] <TeTeT> Once you've been through all the steps and you want more EBS based instances of the same image, simply repeat from step 11 boot_vol.
  61 [17:19] <TeTeT> With this you should have virtualized servers in your UEC within a few minutes, quite a nice time for provisioning.
  62 [17:19] <TeTeT> Especially useful might be to assign permanent public addresses to those instances.
  63 [17:19] <TeTeT> This can be done with help of euca-allocate-address and euca-associate-address.
  64 [17:20] <TeTeT> any questions so far? Everything crystal clear?
  65 [17:21] <TeTeT> Well, it was a very short session then I fear...
  66 [17:22] <TeTeT> Closing words, we're looking into automating the storage of the Ubuntu image on the EBS volume to make this step less work intense. So keep an eye on the Launchpad project.
  67 [17:22] <TeTeT> In the end you should be able to use any Ubuntu image within UEC on Ubuntu 10.04 LTS on an EBS volume in a few minutes, rather than hours
  68 [17:23] <TeTeT> if there's anyone interested in contributing to the project or UEC in general, please get in touch with kim0
  69 [17:23] <TeTeT> we're looking for people with any skills, from coding to writing documentation
  70 [17:24] <TeTeT> we'd also love to hear from you if you try UEC persistency and it works or doesn't work for you
  71 [17:24] <TeTeT> I tested the exercise a few times, but you'll never know
  72 [17:25] <TeTeT> you can touch base with us in #ubuntu-cloud and myself also in #ubuntu-training
  73 [17:25] <ClassBot> obino asked: have you even RAIDed the EBS volumes? Is there any advantage in doing so?
  74 [17:26] <TeTeT> nope, I've never RAIDed the EBS volumes, but would think there might be a bit of a performance hit due to the ATA over Ethernet protocol
  75 [17:26] <TeTeT> also keep in mind that while served via network, the EBS volumes are likely to come from the same Storage Controller (SC)
  76 [17:27] <TeTeT> so not sure if this is a good approach
  77 [17:28] <TeTeT> might be interesting to use drbd though, and maybe use the EBS volume as well as the ephemeral storage and see how that goes
  78 [17:28] <TeTeT> I guess there's quite a bit of room for experimentation for EBS based instances
  79 [17:30] <TeTeT> guess you have no 30 minutes left in the session, so enough time to actually do the exercise if you have a UEC ready :)
  80 [17:30] <TeTeT> no=now
  81 [17:32] <TeTeT> thanks for attending, catch me in #ubuntu-training if you run into problems with the exercise, bye now
  82 [17:33] <kim0> So we finished a bit early on this session
  83 [17:33] <kim0> Daviey starts in less than 30 mins with a puppet session
  84 [17:34] <kim0> Time for a coffee break :)
  85 [17:51] <ClassBot> There are 10 minutes remaining in the current session.
  86 [17:56] <ClassBot> There are 5 minutes remaining in the current session.