== Open Week -- Intro to Ubuntu and Cloud, running your 1st Ubuntu Server on EC2 -- AhmedKamal -- Wed, Oct 13 == {{{#!IRC [17:01] Logs for this session will be available at http://irclogs.ubuntu.com/2010/10/13/%23ubuntu-classroom.html following the conclusion of the session. [17:01] Good Morning, Good Afternoon and Good evening folks [17:01] Let me introduce myself .. This is Ahmed Kamal [17:01] I am the Ubuntu cloud liaison [17:02] So this session is cloud computing basics [17:02] Basically .. it introduces what cloud computing is all about [17:02] I do hear a lot from people who wonder what this cloud thing is all about [17:03] and what's in it "for me" [17:03] this session is exactly for that .. If you're more advanced however, I'd like to invite you to also attend the next session [17:03] by Dustin kirkland, a developer on the Ubuntu server team [17:04] This session is two parts [17:04] part1: I give an intro to cloud, and we have a Q+A session [17:04] part2: I will demo launching your first Ubuntu server VM on the Amazon EC2 cloud [17:05] which will include using cloud-init, a unique Ubuntu server package that makes it easier to work with VMs in the cloud [17:05] So the first thing you need to know about cloud computing .. is that everyone who knows about it has a different view on what it really is [17:06] i.e. every person has some definition, which I'm sure many other quite knowledgeable people would disagree about [17:06] If you're using Gmail/Facebook/Dropbox .. you're already using cloud computing [17:06] and you probably don't even know it! [17:06] I won't try to present a conclusive definition of cloud computing here, but I will say any cloud vendor should provide the following properties [17:07] -Pay per use [17:07] -Instant Scalability [17:07] -Security (If you trust your provider) [17:07] -Reliability [17:07] -APIs [17:07] let me quickly comment on each of those points [17:07] the first .. pay per use, really means that you only pay for what you use [17:08] this is as opposed to a more traditional VPS (Virtual Private Server) [17:08] where you rent by the month .. [17:08] in the pay per use model .. you can use 100 servers for 1 hour and only pay for that! [17:08] -Instant Scalability [17:08] This one means .. that most cloud providers [17:09] do not limit your scaling .. ie if you need more storage or more servers [17:09] you can go ahead and just get them .. no need for pre-committment [17:09] -Security (If you trust your provider) [17:09] Well .. a major debate point [17:09] however .. Some people will never trust their data on someone else's data-center [17:10] however .. some other people will say ... [17:10] just as you trust the bank with your money .. trust us with your data [17:10] it really depends on how valuable your data is to you, how much you trust the provider [17:11] however .. if you have no dedicated IT staff .. throwing your load on a public cloud may even provide better security than running things in house [17:11] -Reliability [17:11] Most cloud vendors will invest in building a reliable data center [17:11] replicated storage [17:11] fault tolerant equipment ...etc [17:11] you gain all of that .. [17:12] -APIs [17:12] The final point .. and it basically means that you control your cloud virtual data center [17:12] through a set of APIs [17:12] APIs are Application Programming Interfaces [17:12] basically .. a way to programmatically create servers, storage, hook them up together [17:13] so you can basically script or program everything that builds your entire virtual data-center [17:13] "automation" is thus big in the cloud world [17:13] and APIs do enable that [17:13] when you're managing hundreds or thousands of virtual servers .. automation is generally important [17:14] so .. [17:14] In essence when you decide to use cloud computing, you're outsourcing your IT services (storage, infrastructure, ...etc) to a vendor that uses his skills to provide a reliable,secure service and uses the economies of scale to provide them at very competitive prices [17:14] Any questions so far ? [17:15] jack002 asked: What happens if vendor stop supporting a cloud? [17:16] The question is what about if my cloud vendor dies/disappears/goes-out-of-business [17:16] A very valid concern indeed [17:16] that is why most cloud concerned IT people [17:16] try to avoid "vendor lock in" [17:16] which means having a standard API to manage your cloud [17:17] and being able to quickly and easily migrate your data and infrastructure off to a different cloud [17:17] there is a lot of effort in that direction [17:17] standardizing APIs .. [17:17] writing wrapper tools like the delta-cloud wrapper [17:17] to standardize cloud management [17:17] however .. the whole scene is too young and fast moving .. so there is not definitive answer yet [17:18] however indeed .. vendor lock in is something you should care about [17:18] Alright [17:18] So, let's move on to the different types of clouds [17:18] Most people categorize clouds as either [17:18] -IaaS [17:19] -PaaS [17:19] -SaaS [17:19] being .. Infrastructure, Platform, Software [17:19] as a Service [17:20] IaaS: Basically means .. you get the low level infrastructure as a service. i.e. what you get is, raw storage, networking bandwidth, load balancers, servers ...etc [17:20] and you build your code and applications on top [17:20] very similar to physical servers really [17:20] This is similar to the Amazon EC2 cloud .. the rackspace cloud and many others [17:20] PaaS: being .. Platform .. you get a "platform" to code against! [17:21] i.e. you don't really see "machine" and "storage" [17:21] you may not even know how many machines your code is running on [17:21] you may not know how the network is configured! [17:21] What you do get [17:21] is a "File-system" for example to put shared data on [17:21] you'd get an API to store/retrieve data from [17:21] ...etc [17:22] How this database you're using is actually configurd .. where does it run .. is none of your concerns! [17:22] The prime example of a PaaS .. is the Google App Engine platform [17:22] The major benefit of PaaS .. is that you don't care/know about the low level stuff [17:23] it all "just works" as far as you're concerned [17:23] however .. the bad part .. is that you don't know the low level stuff also! [17:23] you cannot change anything .. you have almost zero control on the low level configuration ...etc [17:24] Another major benefit, is that you get scaling, for very little or almost no effort [17:24] on the other hand .. with IaaS .. [17:24] you would have to build your application in a scalable manner [17:24] you'd have to use load balancers .. use caching in a smart way .. [17:24] manage DB replicas .. manage DB sharding ... etc [17:24] lots of stuff, which may consume a lot of effort [17:25] The last cloud type we mentioned was [17:25] -SaaS [17:25] basically Software as a service [17:25] What this means .. is that you get a final completed "Software" running in the cloud [17:25] and you just use it! [17:25] examples .. are like Salesforce.com [17:26] It is a complete CRM application [17:26] that is running in the cloud [17:26] you don't need to worry about buying servers, buying storage, running or scaling it [17:26] or backing it up ...etc [17:26] you just pay per use [17:27] the simplicity and benefits are obvious .. while of course vendor lockin is also fairly obvious and problematic [17:27] Cool .. [17:27] Another classification type of Clouds [17:27] is [17:27] private, public, hybrid [17:27] let's quickly comment on that [17:28] -Private: Means a cloud that you built and that you control .. in-house .. behind your firewall .. in your own datacenter [17:28] An enterprise may very well build its own private cloud for its internal users [17:29] -Public: This basically means a cloud that's available publicly on the Internet for anyone that pays [17:29] -Hybrid: This one is a mix of both public and private [17:29] for example .. you may build an inhouse cloud that handles the average load [17:29] however .. a couple of days per month [17:29] you get 5x the normal load for some reason [17:30] so you decide to "burst" out [17:30] to some public cloud [17:30] so you start some virtual servers (VMs) on a public cloud [17:30] perhaps establish a VPN connection between your enterprise and the public cloud [17:30] et voila .. you can expand your virtual datacenter as much as you want to handle the extra load [17:31] so .. any questions so far [17:32] kstailey_ asked: is libcloud an API? [17:32] libcloud is an abstraction library [17:32] that works across many cloud vendors [17:33] so if you're writing an in house script that connects to a public cloud [17:33] creates a few servers and configures them [17:33] if you build your script over libcloud [17:33] you'd be able to quickly replace [17:33] Amazon ec2 for example .. by linode [17:33] across many others [17:33] this is part of the solutions to the vendor lockin problem [17:34] ongolaBoy asked: I guess that all this kind of cloud involve tons of complex hardware and software to run. Am I right ? [17:34] Well .. [17:34] remember when we said public vs private clouds [17:34] One thing I would like to mention is that you can build a private cloud [17:35] today using Ubuntu [17:35] using Ubuntu Enterprise Cloud [17:35] UEC [17:35] that is integrated with the Ubuntu server CDs [17:35] to answer your question about "complexity" [17:35] I would say it is very "simple" for a sysadmin type person to install and get UEC cloud running [17:36] after all everything is easier with Ubuntu :) [17:36] however .. [17:36] of course for production clouds that serve the whole world [17:36] the likes of Amazon and Google data-centers [17:36] indeed the hardware and software involved can become quite involved [17:36] requiring teams of highly skilled unix geeks to keep up and running [17:37] you can start small .. UEC for example can be run on only 2 machines [17:37] or with some hacks .. even just one machine [17:37] and grow your infrastructure as much as you want [17:37] kstailey_ asked: does it make sense to run PaaS on Iaas? [17:38] Good question [17:38] It makes sense to me [17:38] The most popular PaaS platforms are however proprietary [17:38] for example .. the Google App Engine code is closed AFAIK [17:39] if it were open .. you could definitely get it running over IaaS [17:39] after all .. it needs to run over something [17:39] there definitely are some smaller PaaS frameworks that you can start using [17:39] I remember coding against a framework called web2py [17:39] which could either be deployed locally [17:39] OR could be deployed to run over google app engine [17:40] such solutions may be a good starting points to avoid vendor lockin [17:40] komputes asked: This all sounds great but how does one do this virtual datacenter expantion with minimum development knowledge? [17:41] Not sure if "development" in your question means linux sysadmin knowledge or actually development knowledge writing your app [17:42] with APIs .. you basically can create a virtual data-center [17:42] without too much IT infrastructure knowledge [17:42] i.e. without much knowledge of servers, storage, networking ...etc [17:42] an application developer can get by perhaps without a Linux admin at all [17:42] however .. ofcourse you still need app development skills to run your app [17:43] even more so .. to create an app that can be scaled to take advantage of the cloud [17:43] komputes asked: Is there a pre-built solution for using a cloud as a remote desktop middle-man (i.e. I want to support grandma but both of us are behind a firewalls and both ends can't port forward - not admin) [17:44] I don't think you need a cloud for that .. you can use applications like TeamViewer or logmein (free but not foss) [17:44] although you could write your own app if you wanted to of course [17:44] kstailey_ asked: how long before we see Ubuntu switch to OpenStack? [17:45] UEC is built on top of the best open source cloud technologies [17:45] at the moment .. that is Eucalyptus [17:45] openstack is a fine piece of evolving technology though [17:45] it's already in maverick universe .. so you can go ahead and play with it [17:46] we're kinda running out of time [17:46] just wanted to mention a couple of killer cloud use cases [17:46] If your application load varies significantly (top load / least load > 10) [17:46] then you most likely will benefit from running your load over a cloud [17:46] examples [17:46] The Washington Post: 200 EC2 instances (1,407 server hours), convert 17,481 pages of Hillary Clinton’s travel documents within 9 hours [17:47] The New York Times used 100 Amazon EC2 instances + Hadoop application to recognize 4TB of raw  TIFF image into 1.1 million PDFs in 24 hours ($240) [17:47] Okay [17:47] so let's start with a quick demo [17:47] which I hope is going to actually work :) [17:47] In this demo we will use Amazon's EC2 cloud [17:47] we will launch Ubuntu server [17:48] Note that [17:48] if you want to experience Ubuntu server over EC2 cloud yourself [17:48] You can [17:48] For Free ! [17:48] Well actually Canonical pays for you [17:48] You can run it for an hour [17:48] by visiting http://10.cloud.ubuntu.com === [[NEO]] is now known as Neo31 [17:50] The first thing is identifying [17:50] which Ubuntu server you want to run [17:50] you can do so by visiting [17:50] http://uec-images.ubuntu.com/releases/maverick/release/ [17:50] or you can replace maverick by any other release name [17:50] You can pick any AMI you want [17:50] AMI = Amazon Machine Image [17:51] There are 10 minutes remaining in the current session. [17:51] basically a blueprint/template of the virtual server you will start [17:51] assuming you have the ec2 tools setup correctly [17:51] You can create and launch your server with one command like [17:51] ec2-run-instances ami-508c7839 --instance-type t1.micro --region us-east-1 -k MyEC2Key --user-data-file=cloudinit.txt [17:51] ami-508c7839 : Is the AMI we chose [17:52] t1.micro = is the smallest machine Amazon provides [17:52] MyEC2Key: Is my private key (my password if you will) [17:52] cloudinit.txt : [17:52] That is one interesting feature of Ubuntu in the cloud [17:52] cloud-init is a feature which allows the VM to automatically customizes itself [17:53] as it boots [17:53] i.e. you can tell the machine [17:53] to upgrade all software (apt-get upgrade) [17:53] and install apache2 [17:53] all while booting [17:54] Take a look at [17:54] http://paste.ubuntu.com/512497/ [17:54] Which shows the cloud-init file I am using [17:54] This file ensures that [17:54] the VM upgrades itself [17:54] runs byobu [17:54] imports my ssh public key from launchpad! [17:55] automatically so I can login [17:55] installs apache [17:55] and runs some commands at machine boot time [17:55] this is of course awesome [17:55] since all your servers launch from the same template [17:55] you need a way to customize them [17:55] into their respective roles [17:55] and cloud-init does just that [17:55] There are 5 minutes remaining in the current session. [17:56] If you visit [17:56] http://typewith.me/Eip1j21bJ3 [17:56] I will paste the command output interactively [17:57] The bold text ,, is called the machine ID [17:57] Now my server has been created [17:58] I ran describe instances [17:58] this gets me the public name [17:58] of my server [17:58] if you visit that [17:58] there should be apache [17:58] http://ec2-184-72-172-83.compute-1.amazonaws.com/ [17:58] Indeed .. it works! [17:58] that's how easy it is [17:59] to deploy an ubuntu server with a customized apache running on it [17:59] sorry folks if we're running out of time [18:00] one final thing I wanna show you [18:00] is the logs of the machine [18:00] http://paste.ubuntu.com/512499/ [18:00] notice how cloud-init has customized the server === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || UOW Survey: http://is.gd/fZlwL || Event: Ubuntu Open Week - Current Session: Deploying Web Applications in the Cloud - Instructors: kirkland [18:01] upgraded it and installed apache }}}