Virtualisation

Ubuntu Open Week - Virtualisation - Soren Hansen - Wed, Nov 5th, 2008

(04:00:52 PM) soren: Hi, everyone.
(04:01:09 PM) soren: Welcome to the obligatory virtualisation talk.
(04:01:17 PM) soren: It wouldn't be Ubuntu OpenWeek without it, would it?
(04:02:08 PM) soren: I have to say this session is as much for my sake as it is for yours. I'm hoping to get some good feedback from you about how you're using it and what your problems are.
(04:02:34 PM) soren: Or perhaps even better: Some input on why you're not using virtualisation. That would be interesting.
(04:02:39 PM) soren: So, to kick things off.
(04:02:41 PM) soren: ..
(04:02:54 PM) soren: New stuff in Intrepid:
(04:03:15 PM) soren: Intrepid didn't see huge, world changing updates in the basic virtualisation stack.
(04:03:24 PM) soren: The virtualisation stack in my mind consists of:
(04:03:24 PM) soren: kvm
(04:03:28 PM) soren: libvirt
(04:03:30 PM) soren: virt-manager
(04:03:35 PM) soren: virtinst
(04:04:32 PM) soren: They all got updated to more recent upstream versions, which gave us a few new features, such as fully swappable kvm guests and a few other tidbits.
(04:04:49 PM) soren: In intrepid, we also completely rewrote our vm builder.
(04:05:22 PM) soren: If you're not familiar with vmbuilder, it's basically a tool that will create a virtual machine for you in a fraction of the time it would take you to complete a regular install.
(04:06:02 PM) soren: If we have time at the end of this session, I can do a demonstration.
(04:06:51 PM) soren: Just shoot questions in #ubuntu-classroom-chat when you have them, by the way. Don't wait until the end.
(04:07:37 PM) soren: In case you're just tuning into the whole virtualisation buzz, let me explain a bit about what it's good for.
(04:08:16 PM) soren: Actually, virtualisation comes in many shapes and varieties. We've chosen to focus on what is called full virtualisation, which is what kvm provides.
(04:09:12 PM) soren: kvm is a piece of software that runs on your regular system, but looks like a completely separate computer. You can install any operating system in it you like. Windows, other Linux distros, other versions of Ubuntu.
(04:09:28 PM) soren: Anything that you can do to a regular PC, you can do to one created by kvm.
(04:09:52 PM) soren: This is by far the most versatile form of virtualisation.
(04:10:31 PM) soren: If you're a developer, you can set up other operating systems to test interoperability between different things...
(04:10:54 PM) soren: ...you can have other versions of Ubuntu running in it to see if a bug you found is a regression from an older version of Ubuntu..
(04:11:05 PM) soren: ..or even check if a bug has been fixed in a later version.
(04:11:22 PM) soren: You can use it as a "clean room" for doing various tests that might hose your system.
(04:11:38 PM) soren: Or you can use it in production environments to keep services separate.
(04:11:45 PM) soren: The sky is the limit!
(04:11:56 PM) soren: 21:08:33 < ~MadsRH> soren -> The vmbuilder is setup in the terminal? Are there any plans for a GUI?
(04:12:18 PM) soren: VMBuilder is really a library, but also comes with a command line tool to use it.
(04:12:24 PM) soren: ..this is what most people will use.
(04:12:42 PM) soren: However, it was designed to be a library so that it could be easily embedded into other Python projects.
(04:13:04 PM) soren: Perhaps virtinst could use VMBuilder as an alternative form of OS installation.
(04:13:13 PM) soren: ...or it could be exposed directly in virt-manager.
(04:13:29 PM) soren: ...or a web frontend could be used to build VM's.
(04:13:42 PM) soren: Anything is possible. All we need are ideas and code monkeys :)
(04:13:57 PM) soren: 21:11:22 < knome> QUESTION: How effienct kvm is compared to, let's say, VMWare?
(04:14:31 PM) soren: At the risk of offending people who've worked on either side optimizing particular things in each of them, I'd say they're roughly comparable.
(04:14:38 PM) soren: With a twist, though:
(04:15:16 PM) soren: kvm - as you may know - only runs on newer hardware, which has virtualisation features built into the CPU. VMWare runs on any PC.
(04:15:47 PM) soren: If you run VMWare on a PC without virtualisation extensions, I'm quite sure kvm will outperform it somewhat. This is due to the nature of that type of virtualisation.
(04:16:16 PM) soren: VMWare can take advantage of those extensions, too, though, so on the same hardware, they'll be on par.
(04:16:50 PM) soren: 21:06:36 < YokoZar> QUESTION: I'm interested in shipping a VM appliance for using an application with a graphical interface.  It's very easy to do a minimal  VM-based installation using the Ubuntu-server CD, but it's not at all clear to me how I get a minimal desktop suitable for a VM appliance.
(04:16:56 PM) soren:  Installing the Ubuntu-destop package, for instance, will install all manner of video drivers that arean't needed.
(04:17:55 PM) soren: For starters, a VM appliance (or virtual appliance) is a virtual machine that you distribute to clients or customers that performs a  single task.
(04:18:34 PM) soren: The idea is that you preconfigure it to do a few things really well, and usually wrap it all up in a fancy wrapping.
(04:19:11 PM) soren: A colleague of mine has written a HOWTO about creating such an appliance.
(04:19:35 PM) soren: ...it even ended up in Linux Mag: http://www.linux-mag.com/id/4829
(04:20:19 PM) soren: It's an area where we still have a lot of work to do. it could certainly do with a better frontend for deployment..
(04:20:44 PM) soren: ..such as setting it up in the end user's network, perhaps hooking it into whatever other services it needs to interact with, etc. etc.
(04:21:07 PM) soren: If anyone has the time to work on something like that, please do get in touch.
(04:21:25 PM) soren: 21:07:29 < alexharrington> QUESTION: What does "fully swappable" mean in relation to a kvm guest?
(04:22:24 PM) soren: Not too long ago, if you had a kvm guest running that used 2 GB of memory, those 2 GB of RAM on the host was stuck in RAM and couldn't be swapped out to disk.
(04:22:41 PM) soren: This is no longer true.
(04:22:55 PM) soren: This is actually the primary reason the /dev/kvm device wasn't open to everyone.
(04:23:37 PM) soren: Anyone with access to that device could pin all your RAM, which was rather unfortunate.
(04:24:06 PM) soren: 21:12:29 < YokoZar> QUESTION: So, is there going to be support for a "minimal desktop" sort of VM appliance sometime?  Like another package seed?
(04:24:31 PM) soren: I don't really think a graphical environment in an appliance is something we want to put a lot of effort into.
(04:25:19 PM) soren: I'm much more inclined to go the way of your garden variety router which has a web interface for configuring everything.
(04:25:21 PM) soren: Graphical environments are just a hassle :)
(04:25:54 PM) soren: 21:13:13 < Zizou> Question: which is the adventaje in kernel based virtualisation comparing to software virtualisation?
(04:26:30 PM) soren: I'm not entirely sure what you mean by software virtualisation.. If you mean full virtualisation, but done entirely in software, like qemu does it, the main advantage is speed.
(04:26:43 PM) soren: Emulation is a slow, slow process.
(04:27:23 PM) soren: If you're thinking more along the lines of jails or containers, the main advantage is versatility.
(04:27:41 PM) soren: You can't run Windows in a jail or a container. You need something the pretends to be a complete PC so that Windows doesn't get all confused. :)
(04:27:52 PM) soren: 21:13:37 < Israphel> QUESTION: in the setup of my mother there's a virtualization option to enable, should I enable it?
(04:28:20 PM) soren: It all depends.
(04:29:05 PM) soren: Perhaps your mom needs to run a few Windows apps even though she has switched to Ubuntu. Instead of going the way of dual booting, you could enable virtualisation on her machine, set her up with kvm and virt-manager and have her run those last few windows apps inside kvm.
(04:29:48 PM) soren: A few years from now, I think we're going to see much more widespread use for virtualisation. In fact, by then it's likely that you'll be using it without even knowing it.
(04:30:16 PM) soren: 21:14:19 < zul> QUESTION: what features does the vmbuilder rewrite has?
(04:30:26 PM) soren: The new VMBuilder features an extensive plugin system.
(04:30:50 PM) soren: This makes it much, much easier to add support for other distros and hypervisors, and even to add entirely new functionality.
(04:31:11 PM) soren: All the libvirt integration it does is contained in a plugin by itself, for instance.
(04:31:43 PM) soren: It's also a lot more friendly to watch, and it's simply much more flexible in many,  many ways.
(04:32:11 PM) soren: 21:14:31 < nealmcb> QUESTION: So just how fast is vm-builder, anyway :)
(04:32:35 PM) soren: I forget the most recent benchmarks I've done, but under optimal conditions, I'm certain it can create a virtual machine in around a minute, probably less.
(04:33:07 PM) soren: I have a few optimisations in my back pocketet that should shave off an additional 5-10 seconds from that benchmark.
(04:33:22 PM) soren: 21:14:35 < ~Mauricio> QUESTION: KVM is installed for default into Ubuntu?
(04:33:51 PM) soren: It's not installed by default, no. It's just a simple apt-get away, though :)
(04:33:54 PM) soren: 21:21:28 < ~johnsgruber> QUESTION: Is there an easy way to tell whether my computer has virtualization extensions?
(04:34:01 PM) soren: An almost sure way to tell is to run this command:
(04:34:17 PM) soren: egrep '^flags.*(vmx|svm)' /proc/cpuinfo
(04:35:03 PM) ntinos_19 is now known as ntinos_
(04:35:07 PM) soren: If that gives you any output, your cpu has the right extensions. It might still need to be enabled in the BIOS, but installing kvm will attempt to run, and should detect if that is the case.
(04:35:31 PM) soren: QUESTION: Regarding the minimal desktop, what about people who are using something like vmware on their desktop to test things?  It seems  like a lot of people try out Ubuntu this way, and it'd be nice to have it snappier there.
(04:35:43 PM) soren: I'm not sure I understand this use case? Are we still talking about appliances?
(04:35:57 PM) soren: 21:30:31 < alexharrington> QUESTION: Does/will kvm support direct access to specific host hardware - eg a TV tuner
(04:36:19 PM) soren: Not right now, but it will come.
(04:36:30 PM) soren: 21:31:52 < YokoZar> QUESTION: I hate to ask, but any interest in a graphical front-end to vmbuilder?
(04:36:47 PM) soren: I think my money is on the web frontend.
(04:37:28 PM) soren: That way, we can distribute the webified version of it as a vm, and people can run it on their windows systems or fedora systems or whatever and create even more Ubuntu based virtual machines in a snap.
(04:38:23 PM) soren: Now, if you want to try out kvm, what you should do is install a metapackage we've created for it..
(04:38:51 PM) ***soren tries to find it..
(04:39:31 PM) soren: ubuntu-virt-mgmt
(04:39:48 PM) soren: Start by installing that: sudo apt-get install ubuntu-virt-mgmt
(04:40:15 PM) soren: Oh, and ubuntu-virt-server, too.
(04:40:22 PM) soren: Same deal: sudo apt-get install ubuntu-virt-server
(04:40:42 PM) soren: When that's done, you have virt-manager, which is the graphical frontend for (among other things) kvm.
(04:41:03 PM) soren: You have libvirt, which is a handy wrapper and management library for kvm, Xen, openvz, etc.
(04:41:13 PM) soren: You have kvm which is the core "hypervisor".
(04:41:27 PM) soren: And you have a few other things that comes along with these packages.
(04:41:37 PM) soren: First, you want to add yourself to the libvirtd group.
(04:41:59 PM) soren: The short explanation is that it allows you to do stuff with your network config that you *really* don't want to be without.
(04:42:38 PM) soren: As with any other new group membership, you need to log out and log back in to make it take effect.
(04:43:05 PM) soren: If you don't want to do that right now, that's fine.
(04:43:10 PM) soren: Just do it later. You'll be thankful :)
(04:43:29 PM) soren: Fire up virt-manager, and you should see an overview screen.
(04:43:42 PM) soren: Virt-manager, IIRC, is in the Applications->System menu.
(04:44:22 PM) soren: 21:41:13 < maluta> QUESTION: Xen has any future in newer versions os Ubuntu?
(04:44:42 PM) soren: Xen has finally started to slowly trickle into the upstream vanilla kernels.
(04:44:54 PM) soren: We're obviously not going to actively keep it away from our kernels then :D
(04:45:08 PM) soren: ...so in fact, the server kernel runs as a Xen domU without any modifications.
(04:45:29 PM) soren: Later versions will probably work as a dom0, when that lands upstream.
(04:45:37 PM) soren: I don't know when that's likely to happen, though.
(04:46:07 PM) soren: We'll probably not spend a whole lot of time on Xen going forward, though. We chose to focus on kvm a while ago, and we're quite happy with that decision.
(04:46:43 PM) soren: 21:42:23 < ~toobuntu> QUESTION: With ubuntu-vm-builder, we had to --addpkg foo --addpkg bar, listing every pkg separately.  Does the python rewrite accept  multiple pkgs in a comma separated list or something a little simpler?
(04:47:15 PM) soren: I believe that's still the same. I've never considered this a problem, really.
(04:47:27 PM) soren: For me, I usually only install one or two extra packages.
(04:48:00 PM) soren: The way I've set up my production environment (I host a bunch of stuff for myself, some friends, etc), is with one virtual machine per service.
(04:48:35 PM) soren: I have a virtual machine for mysql, one for apache, one for kannel, one for nagios, one for postfix/dovecot..
(04:48:56 PM) soren: Feel free to file a bug about this, though. I've never thought it was an issue :)
(04:49:07 PM) soren: 21:43:37 < Zizou> QUESTION: Are those packages available in Debian? maybe with other names...
(04:49:16 PM) soren: Those metapackages are not in Debian, no.
(04:49:40 PM) soren: In Debian, just install libvirt-bin, virt-manager, and kvm, and you should be well on your way.
(04:49:49 PM) soren: 21:45:25 < ~foob9> QUESTION: is there a howto available?
(04:50:16 PM) soren: For getting virt-manager and all that running? Probably :) I don't know to be honest.
(04:50:57 PM) soren: Ok, you have virt-manager installed and you're looking at the overview. Double click on the line that reads "localhost (system)" or "localhost (user)" depending on whether you're a member of libvirtd.
(04:51:05 PM) soren: You're now connected to the hypervisor.
(04:51:24 PM) soren: Click the "New" button, and you're on your way to setting up your first virtual machine.
(04:52:00 PM) soren: It should be mostly self explanatory. If not, don't hesitate to bother us in the #ubuntu-virt channel.
(04:52:08 PM) soren: Or in #ubuntu if you're more comfortable there.
(04:53:00 PM) soren: Apart from the obvious "I don't have a computer that supports kvm" what other reasons do you have for not using virtualisation?
(04:53:11 PM) soren: Is it too difficult to get started?
(04:53:20 PM) soren: Do you not see the point?
(04:53:42 PM) soren: I'm very interested in this, so that we can try to fix the problems.
(04:54:35 PM) soren: 21:54:04 < alexharrington> soren: Initially I was mainly concerned about disk io throughput - especially in things like proxy servers and mail servers. KVM is  working well though.
(04:54:49 PM) soren: I/O performance has been a concern for many people.
(04:55:00 PM) soren: Network performance in KVM is IMO stellar.
(04:55:35 PM) soren: I've moved files from my desktop machine to a virtual machine running on it at 180 MB/s.
(04:55:44 PM) soren: That's not too shabby :)
(04:56:10 PM) soren: Disk I/O has been massively improved in recent kvm versions.
(04:56:33 PM) soren: You'll see a dramatic performance increase in kvm in Jaunty.
(04:56:50 PM) soren: 21:56:04 < alexharrington> QUESTION: any news on paravirtualised block devices - I know there is a paravirt network driver now (which we're using on a couple of  guests).
(04:57:14 PM) soren: The 180 MB/s was with paravirt network drivers. They're called virtio devices in kvm.
(04:57:50 PM) soren: If you use virt-manager to install Ubuntu (and choose Ubuntu in the OS selection drop down), you'll have virtio networking by default.
(04:58:13 PM) soren: There's also virtio block devices, but it doesn't make a lot of difference right now, actually.
(04:58:16 PM) jcastro: (2 minutes left!)
(04:58:22 PM) soren: ...because that's not where the bottleneck is.
(04:58:48 PM) soren: ..but with recent kvm versions, it's likely to make a lot of difference, if you have a storage backend that can keep up.
(04:58:55 PM) soren: Final questions?
(05:00:02 PM) soren: Alright, let's call it a day then. Thanks for showing up, everyone.

MeetingLogs/openweekintrepid/Virtualisation (last edited 2008-11-05 22:14:57 by pool-70-16-60-167)