Ubuntu Wiki

   1 [19:59] <kim0> Next Up is OpenStack Intro session
   2 [19:59] <soren> o/
   3 [20:00] <soren> kim0: How does it work? Do you copy questions from somewhere else or do I need to do that myself?
   4 [20:00] <soren> Or do people just ask here?
   5 [20:00] <kim0> soren: you "/msg ClassBot !q" then !y on every question
   6 [20:01] <kim0> soren: please join #ubuntu-classroom-chat as well
   7 [20:01] <soren> This is complicated :)
   8 === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Event: Ubuntu Cloud  Days - Current Session: Open-Stack Introduction - Instructors: soren
   9 [20:01] <soren> Hello, everyone!
  10 [20:01] <ClassBot> Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/23/%23ubuntu-classroom.html following the conclusion of the session.
  11 [20:02] <soren> I'm Soren, I'm one of the core openstack developers.
  12 [20:02] <soren> OpenStack consists of two major components and a couple of smaller ones.
  13 [20:02] <soren> The major ones are OpenStack Compute, codenanmed nova.
  14 [20:02] <soren> ...and OpenStack Storage, codenamed Swift.
  15 [20:03] <soren> Swift is what drives Rackspace Cloud Files, which is a service very much like Amazon S3.
  16 [20:03] <soren> It's *massively* scalable, and is used to store petabytes of data today.
  17 [20:03] <soren> I work on Nova, though, so that's what I'll spend most time talking about today.
  18 [20:03] <soren> Nova is a project that started at NASA.
  19 [20:04] <soren> Apart froms ending stuff into space, NASA also does a bunch of other research things for the US government.
  20 [20:04] <soren> AMong them: "Look into this cloud computing thing"
  21 [20:04] <soren> This is what turned into the NASA Nebula project.
  22 [20:05] <soren> If you google it (I forgot to do so in advance), you'll find images of big containers that say Nebula on the side.
  23 [20:05] <soren> They're building blocks for NASA's cloud.
  24 [20:05] <soren> Anyways, they started our running this on Eucalyptus.
  25 [20:05] <soren> The same stuff that drives UEC.
  26 [20:06] <soren> This got.. uh... "old" eventually, and they decided to throw it out and write their own thing.
  27 [20:06] <soren> ..so they did, and they open sourced it.
  28 [20:07] <soren> RAckspace had plans for open sourcing their cloud platform, too, so they called NASA and said "wanna play?" (paraphrasing a little bit), and they were up for it.
  29 [20:07] <soren> So Rackspace had Swift, NASA had Nova. We put it together and called it OpenStakc.
  30 [20:07] <soren> OpenStack, even.
  31 [20:08] <soren> If you go to look at them, and they don't look like two pieces of the same puzzle, this is why. They share no ancestry, really.
  32 [20:08] <soren> They now work happily together, though.
  33 [20:08]  * soren attempts to work that qeustions thing
  34 [20:09] <ClassBot> EvilPhoenix asked: What exactly IS Open-Stack?
  35 [20:09] <soren> I guess that one is answered..
  36 [20:09] <ClassBot> medberry asked: Can you briefly differentiate openstack from eucalyptus
  37 [20:09] <soren> Yes. Yes, I can.
  38 [20:10] <soren> So, Eucalyptus corresponds to Nova.
  39 [20:10] <soren> They both focus on the compute side of things, while providing a *very* simple object store. Neither try to do any sort of large scale stuff.
  40 [20:10] <soren> Err..
  41 [20:10] <soren> For storage, I mean.
  42 [20:11] <soren> For the compute part, the architectures are *very* dissimilar.
  43 [20:11] <soren> So, last I looked (admittedly 1½ year ago, but I'm told this is still true), Eucalyptus is strictly hierarchical.
  44 [20:12] <soren> There's one "cloud controller" at the top.
  45 [20:12] <soren> There's a number of cluster controllers beneath this one cloud controller.
  46 [20:12] <soren> ...and there's a number of "node controllers" beneath the cluster controllers.
  47 [20:13] <soren> Eucalyptus is written in Java, and uses XML and web services for all its communication.
  48 [20:13] <soren> It polls from the top down.
  49 [20:13] <soren> Never the other way around.
  50 [20:14] <soren> Nova uses message queues.
  51 [20:14] <soren> Nova is written in Python.
  52 [20:14] <soren> We have no specific structure that must be followed.
  53 [20:14] <soren> There are a number of components: compute, network, scheduler, objectstore, api, and volume.
  54 [20:14] <soren> There can be any number of each of them.
  55 [20:15] <soren> So Nova itself has no single points of failure.
  56 [20:15] <soren> Oh, Eucalyptus's cluster and node controllers are written in C, by the way. I forgot.
  57 [20:16] <soren> All of Nova is Python.
  58 [20:16] <soren> AFAIK, Eucalyptus supports KVM and Xen.
  59 [20:16] <soren> We support KVM, Xen, Hyper-V, user-mode-linux, LXC (if not now, then *very* soon), VMWare vsphere..
  60 [20:17] <soren> Eerr..
  61 [20:17] <soren> Yeah, I think that's all.
  62 [20:17] <soren> WE also support a number of different storage backends (for EBS-like stuff): iSCSI, sheepdog, Ceph, AoE..
  63 [20:17] <soren> And one more, which I forget what is.
  64 [20:17] <soren> We're very, very modular in this way.
  65 [20:18] <soren> Last I checked, Eucalyptus supported AoE. They may or may not support more now. I'm not sure.
  66 [20:18] <ClassBot> kim0 asked: I understand openstack focuses on large scale deployments .. How suitable is it for openstack to be deployed in a small setting (5 servers?)
  67 [20:18] <soren> I'm glad you asked.
  68 [20:19] <soren> The Ubuntu packages I made of Nova work out-of-the-box on a single machine.
  69 [20:20] <soren> Scaling it out to 5 servers shouldn't be much work. There's some networking things that need to be set up, you need to point it at a shared database (so far, we're working towards a completely distributed data store) and a rabbitmq server.
  70 [20:20] <soren> We're suffreing a bit from our flexibility, really.
  71 [20:21] <soren> We can make very few assumptions about people's set up, so there might be a number of things that need to be set up correctly (e.g. which ip to use to reach an api server (or a load balancer in front of them)), which server to use for this, whcih server to use for that).
  72 [20:22] <soren> It's pretty obvious pretty quickly, though, if something isn't pointed the right way.
  73 [20:22] <soren> We're "blessed" with a team of people in Europe and in most US timezones, so if you run into trouble #openstack (irc channel) is open almost 24/7 :)
  74 [20:23] <ClassBot> kim0 asked: Is nova deployed at rackspace in production yet ? did you guys go with xen or kvm, and why ?
  75 [20:23] <soren> Nova is not in production at Rackspace yet, no.
  76 [20:23] <soren> Rackspace has an existing platform with which we've not completely hit feature parity.
  77 [20:23] <soren> ...and apparently, it's not ok to make Rackspace's customers suffer because we want to run a different platform :)
  78 [20:23] <soren> Rackspace will be using Xen Server.
  79 [20:24] <soren> Oh, I forgot to list that as a supported hypervisor. It is.
  80 [20:24] <soren> That's what they're used to, and that's what they can get support for for running Windows and stuff.
  81 [20:24] <ClassBot> markoma asked: Gluster was mentioned in a previous discussion. Is swift the right way to go, or Gluster?
  82 [20:25] <soren> They do very different things.
  83 [20:25] <soren> Gluster aims to provide a POSIX compliant filesystem.
  84 [20:25] <soren> Swift is an object store.
  85 [20:25] <soren> You address full objects. You cannot seek back and forth, replace parts of objects, etc.
  86 [20:26] <soren> Very much like Amazon S3.
  87 [20:26] <soren> Gluster recently announced they want to contribute to Swift. I don't know exactly how, but something's afoot :)
  88 [20:26] <ClassBot> jrisch asked: I think it's still unclear from the documentation, but it mentions something about a cloudpipe vm, but doesn't clarify it's role nor it's usage. Can you elaborate on that?
  89 [20:26] <soren> Ah, yes.
  90 [20:27] <soren> Cloudpipe is something NASA uses.
  91 [20:27] <soren> I don't think anyone else does, and perhaps will.
  92 [20:27] <soren> Each project has its own private subnet assigned.
  93 [20:27] <soren> Typically in the 10.0.0.0/8 range.
  94 [20:27] <soren> It's not reachable from the internet.
  95 [20:27] <soren> Cloudpipe images are images with an openvpn server in them.
  96 [20:28] <soren> Each project has such an instance running. They can connect to it using openvpn, and they can then reach their instances.
  97 [20:28] <soren> It's not required at all.
  98 [20:28] <soren> I've never used it.
  99 [20:28] <ClassBot> topper1 asked: is rabbitmq a SPOF since its clustering doesn't replicate queues?
 100 [20:29] <soren> In a sense.
 101 [20:29] <soren> From Nova's point of view, it's a bit of a black box.
 102 [20:30] <soren> We speak to something that speaks AMQP. We expect it to behave.
 103 [20:30] <soren> Just like we use an SQL database of some sort and expect it to behave.
 104 [20:31] <soren> RabbitMQ is way more stable than what we could have hacked up in the time it took to run "apt-get isntall rabbitmq-server".
 105 [20:31] <soren> *way* more stable.
 106 [20:32] <soren> There's work in progress to build a queueing service for OpenStack, but in general, we try to use existing components.
 107 [20:32] <ClassBot> n1md4 asked: There seems to be install guides for CentOS, RHEL, and Ubuntu, is there nothing specifically for Debian?
 108 [20:32] <soren> Not right now, I don't think.
 109 [20:33] <soren> I'd be *thrilled* if a DD stepped up and put OpenStack into Debian.
 110 [20:33] <soren> ...and sorted out all the dependencies.
 111 [20:33] <soren> It's silly not to, really.
 112 [20:33] <soren> It's just that noone has done it yet.
 113 [20:33] <ClassBot> markoma asked: do you, would you, use Ensemble to manage services for OpenStack?
 114 [20:34] <soren> I've no clue about what Ensemble does at the moment, so I can't really answer that.
 115 [20:34] <ClassBot> jrisch asked: If cloudpipe isn't required, how do you set up access to the VM's, IP mappings and stuff. Do the physical node act as a pipe/NAT device?
 116 [20:34] <soren> I tend to use floating ip's.
 117 [20:34] <soren> They're public IP's that you can dynamically assign to instances.
 118 [20:35] <soren> Alterntively, you can just use one of the other netowrk managers and use a subnet that's routable.
 119 [20:35] <ClassBot> jrisch asked: So if you speak AMQP to the message queue, could one use ActiveMQ instead? (it supports clustering as far as I know).
 120 [20:35] <soren> AFAIK, we don't do anything that requires RabbitMq.
 121 [20:36] <soren> So I guess ActiveMQ would work, if it speaks AMQP.
 122 [20:36] <ClassBot> topper1 asked: Is there work afoot to create API documentation (rest api) for swift... right now it requires 'you read the python')
 123 [20:36] <soren> Uh, there's plenty of docs.
 124 [20:36] <soren> HAng on.
 125 [20:37] <soren> http://www.rackspace.com/cloud/cloud_hosting_products/files/api/
 126 [20:37] <soren> Same thing.
 127 [20:37] <soren> I don't know where the ones labeled "openstack" are, but it's the same thing.
 128 [20:38] <soren> Ah, question queue is empty..
 129 [20:38] <soren> Where was I? :)
 130 [20:38]  * soren scrolls up
 131 [20:38] <soren> Nowhere, apparantly.
 132 [20:38] <soren> Ok, process..
 133 [20:38] <soren> We do time based releases.
 134 [20:39] <soren> Just like Ubuntu.
 135 [20:39] <soren> Except we have 3 months cycles, rather than 6 months.
 136 [20:39] <soren> We align with Ubuntu so that every other OpenStack release should almost coincide with an Ubuntu release.
 137 [20:40] <soren> We have feature freezes, beta freezes, RC freezes and final freezes just like Ubuntu.
 138 [20:40] <soren> This is no coincidence :)
 139 [20:40] <soren> Ubuntu is our reference platform.
 140 [20:41] <soren> I'm a core dev of Ubuntu, too, so if we have an problem with a component outside Nova, we can fix it and get it into our reference platform quite easily.
 141 [20:41] <soren> This holistic view of the distribution has served us very well, I think.
 142 [20:42] <soren> Nova can be way cool, but if there are bugs in libvirt, we're going to suffer, too, for instance.
 143 [20:42] <soren> Ok, so say you wanted to work on something in Nova (or other parts of Openstack).
 144 [20:43] <soren> You can branch the code from launchpad (which we use for everything: blueprints, bugs, code, answers) using "bzr branch lp:nova"
 145 [20:43] <soren> Hack on it, upload a branch to launchpad, and click the "propose for merge" button.
 146 [20:43] <soren> Within a couple of days someone should have looked at it and reviewed id.
 147 [20:44] <soren> it.
 148 [20:44] <soren> If it's good, it gets approved. If it's less good, we (try to) give constructive feedback so that you can fix ti.
 149 [20:44] <soren> Once it's good, it's approved.
 150 [20:44] <soren> Once approved, a component called Tarmac takes over.
 151 [20:44] <soren> Tarmac is run from our Jenkins instance: http://jenkins.openstack.org/
 152 [20:45] <soren> It looks for approved branches on Launchpad, merges them, and runs our test suite.
 153 [20:45] <soren> We have around 75% code coverage, I think.
 154 [20:45] <soren> Far from ideal, but it cathces quite a few things.
 155 [20:45] <soren> If the tests pass, your branch is merged.
 156 [20:45] <soren> And that's it.
 157 [20:46] <soren> If the tests fail, your branch gets set back to "needs review" and you can go and fix it again.
 158 [20:46] <soren> This is fine. It happens all the time. Don't sweat it.
 159 [20:46] <soren> WE're also doing some integration tests.
 160 [20:46] <soren> Oh, one other thing:
 161 [20:47] <soren> When a patch gets merged, it triggers a package build.
 162 [20:48] <soren> This means that if Launchpad doesn't have a huge backlog, less than 20 minutes after your branch has been reviewed, you can "apt-get upgrade" and get a fresh version of Nova with your patch in it.
 163 [20:48] <soren> So we continuously test that our packages build.
 164 [20:48] <soren> I have a Jenkins instance that checks the PPA for updates.
 165 [20:48] <soren> If there are updates, it installs the updates and runs a bunch of integration tests.
 166 [20:49] <soren> So within... I dunno, 35 minutes or so, probably, your patch has gone through unit tests, packages builds, and integration tests.
 167 [20:49] <soren> I think that's pretty cool.
 168 [20:50] <soren> We're working on expanding these tests.
 169 [20:50] <soren> So that we test more combinations of stuff.
 170 [20:51] <soren> I currently test KVM with the EC2 API using iSCSI volumes on Lucid, Maverick, and Natty.
 171 [20:51] <soren> We provide backported versions of stuff that is needed to run Openstack on Lucid, which we do support.
 172 [20:51] <ClassBot> There are 10 minutes remaining in the current session.
 173 [20:51] <soren> ...as well as Maverick and NAtty.
 174 [20:51] <soren> Well, there's nothing backported for Natty, because we put that directly into UBuntu.
 175 [20:52] <ClassBot> kim0 asked: Can you talk a bit about nova's roadmap
 176 [20:52] <soren> Sort of.
 177 [20:52] <soren> There are some things on the road map already.
 178 [20:53] <soren> ...but we have a design summit coming up, where we'll be talking much more about the roadmap.
 179 [20:53] <soren> It's an open event in Santa Clara in about a month, if anyone wants to come.
 180 [20:53] <soren> Should be fun.
 181 [20:53] <soren> Things that I do know on the road map already:
 182 [20:54]  * soren looks desperately for the list.
 183 [20:55] <soren> https://blueprints.launchpad.net/nova
 184 [20:55] <soren> Well, this is the list of everything.
 185 [20:55] <soren> Cactus is the release we're working on now.
 186 [20:55] <soren> Bexar is the previous one.
 187 [20:55] <soren> Diablo the next one.
 188 [20:56] <soren> Lots of different companies work on OpenStack. They have their own priorities.
 189 [20:56] <soren> Whatever they want to work on, they can.
 190 [20:56] <ClassBot> There are 5 minutes remaining in the current session.
 191 [20:56] <soren> So in that respect, it's hard to say what's going to land at any given time. It depends on what people feel like working on.
 192 [20:57] <soren> We're going to split out some stuff from nova (volume and network services), though.
 193 [20:57] <soren> That seems pretty certain right now.
 194 [20:57] <soren> And add support for the EC2 SOAP API.
 195 [20:57] <soren> People keep telling me no-one uses it, but... meh. I want to add it.
 196 [20:58] <soren> MAn, I can't really remember more stuff right now :(
 197 [20:58] <ClassBot> jrisch asked: I know that Swift is in production several places (other than Rackspace) - do you know of any companies that are using NOVA (besides NASA)...?
 198 [20:58] <soren> Not at the moment, no.
 199 [20:58] <soren> This current dev cycle has been one focused on stability and deployability.
 200 [20:59] <soren> The goal has been to get Nova to a point where people could actually use it in production.
 201 [20:59] <soren> I've blogged a bit about some the stuff I've done on that.
 202 [20:59] <soren> ..but lots of others have worked on it, too.
 203 [20:59] <soren> I guess that's it?
 204 [21:00] <soren> I hope it's been useful.
 205 [21:00] <kim0> Thanks soren
 206 [21:00] <kim0> This has been great
 207 [21:00] <kim0> Thanks everyone ..
 208 [21:00] <kim0> Hope you enjoyed the sessions
 209 [21:01] <kim0> See you tomorrow for the second day
 210 [21:01] <ClassBot> Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/23/%23ubuntu-classroom.html