Ubuntu Wiki

   1 [18:02] <Daviey> My name is Dave Walker (Daviey), and I am a member of the Ubuntu Server Team.
   2 [18:02] <Daviey> Welcome to the puppet classroom session.  This session is mainly targeted at those that have had minimal or no exposure to the puppet.
   3 [18:03] <Daviey> It allows reproducible, consistent deployments, which is good for horizontal scaling and replacing machines which have malfunctioned.
   4 [18:03] <Daviey> A good reference for more details about the project is at:
   5 [18:03] <Daviey> http://projects.puppetlabs.com/projects/puppet/wiki/About_Puppet
   6 [18:03] <Daviey> Please take a few moment to grok the content of that page, there is little point in my reproducing the content here.
   7 [18:03]  * Daviey waits a few minutes.
   8 [18:04] <Daviey> Now, some of that might sound little complicated but it really is simple when you get started.
   9 [18:05]  * Daviey continues.
  10 [18:05] <Daviey> Puppet focuses on the 'configuration' management.  The initial operating system deployment is usually done with either, preseeding the installer, cobbler, FAI or simply spawning a cloud machine, such as EC2.
  11 [18:05] <Daviey> In regards to EC2.. people tend to use user-scripts or increasingly cloud-init.
  12 [18:06] <Daviey> Once the base operating is installed, there is always some changes that need to be made to make the server usable for production.  This varies from performance tweaks, application configuration and even custom versions of packages.  This could all be handled with scripts and such, but this is less than clean and near impossible to maintain.  This is where puppet provides a clean solution.
  13 [18:06] <Daviey> Once the base operating is installed, there is always some changes that need to be made to make the server usable for production.  This varies from performance tweaks, application configuration and even custom versions of packages.  This could all be handled with scripts and such, but this is less than clean and near impossible to maintain.  This is where puppet provides a clean solution.
  14 [18:06] <Daviey> Puppet generally acts on a client/server method, to manage multiple nodes.  However, it is also possible to use puppet on a single host.  For simplicity, this session will demonstrate a single host deployment example and some of the features of puppet via their configuration format - called a manifest.
  15 [18:07] <Daviey> In this session, we will do the following:
  16 [18:07] <Daviey> • Connect to an instance in the cloud
  17 [18:07] <Daviey> • Install puppet
  18 [18:07] <Daviey> • Initial configuration
  19 [18:07] <Daviey> • Configure the same node to install and create a basic apache virtual host
  20 [18:07] <Daviey> Firstly, i hope everyone will be able to look at a console window, and this IRC session concurrently.
  21 [18:08] <Daviey> I'm going to invite everyone to connect via ssh to a cloud instance:
  22 [18:08] <Daviey> $ ssh demo@demo.daviey.com
  23 [18:08] <Daviey> You'll need to accept the host key
  24 [18:08] <Daviey> I don't think it really requires verification in this instance.
  25 [18:08] <Daviey> (Although, it's generally good pratice to compare the fingerprint)
  26 [18:09] <Daviey> The password is 'demo'
  27 [18:09] <Daviey> (Secure huh?)
  28 [18:10]  * Daviey waits for a confirmation.
  29 === pvo_ is now known as pvo
  30 [18:10] <Daviey> I will type in the IRC channel comments, so please multi-task by looking at both.. Thanks :)
  31 [18:11] <Daviey> So, i just checked to see if we have apache2 installed... we do not!
  32 [18:11] <Daviey> (You can check there is nothing running as a httpd on port 80, by visiting http://demo.daviey.com
  33 [18:12] <Daviey> (You should get a failure)
  34 [18:12] <Daviey> I'm running sudo apt-get update, to make sure our indexes are updated
  35 [18:12] <Daviey> The observant amongst you, will notice i'm running Natty
  36 [18:12] <Daviey> The current development version
  37 [18:12] <Daviey> (I must be crazy doing a demo on this! :)
  38 [18:13] <Daviey> So, i just, sudo apt-get install puppet
  39 [18:13] <Daviey> This installs the puppet application and it's dependencies.
  40 [18:14] <Daviey> This stage, would normally be done automatically during installation
  41 [18:14] <Daviey> (if you preseed it such)
  42 [18:14] <Daviey> You'll notice the output here:
  43 [18:14] <Daviey> puppet not configured to start, please edit /etc/default/puppet to enable
  44 [18:14] <Daviey> Did you all see the START=no, option
  45 [18:15] <Daviey> This means that the puppet client agent will not run automatically
  46 [18:15] <Daviey> My intention is to invoke puppet manually.. so i do not need the client to be running
  47 [18:16] <Daviey> (one moment please)
  48 [18:17] <Daviey> (slight technical issue, please hold)
  49 [18:21] <Daviey> Okay!
  50 [18:21] <Daviey> we are back
  51 [18:21] <Daviey> okay, this is the directory structure we should see
  52 [18:21] <Daviey> on a fresh installation
  53 [18:22] <Daviey> Okay, i have just copied a manifest to /etc/puppet/manifest
  54 [18:22] <Daviey> I hope everyone can see this
  55 [18:22] <Daviey> It's quite a quick one i have thrown together
  56 [18:22] <Daviey> It should:
  57 [18:22] <Daviey> Install apache2
  58 [18:22] <Daviey> add a virtual host, called demo.daviey.com
  59 [18:23] <Daviey> and enable it
  60 [18:23] <Daviey> (I'll make it avaliable afterwards via a pastebin)
  61 [18:23] <Daviey> The stanza towards the bottom mentions, ip-10-117-82-138
  62 [18:23] <Daviey> (for the observant, you'll notice this is the hostname of the machine)
  63 [18:24] <Daviey> I could equally, have put 'default' here... which would mean that it would do it for every machine connected
  64 [18:24] <Daviey> (in this instance, i am only using one machine)
  65 [18:24] <Daviey> Now, the actual virtual host needs a template...
  66 [18:24] <Daviey> lets create it.
  67 === daker is now known as daker_
  68 [18:25] <Daviey> puppet uses Ruby's ERB template system:
  69 [18:25] <Daviey> You'll notice that there are parts which can be expanded.
  70 [18:25] <Daviey> So, this is a generic apache virtual host template, that could be used for other virtualhosts
  71 [18:25] <Daviey> other than demo.daviey.com
  72 [18:26] <Daviey> Now... lets make puppet do it's thing!
  73 [18:27] <Daviey> I love it when a plan comes together.
  74 [18:28] <Daviey> Essentially, i did a dry run with this configs before the session.. and didn't clean up properly!
  75 [18:28] <Daviey> This is why i should have used puppet to clean up, as it would have done it better than me!
  76 [18:29] <Daviey> So, puppet install apache2 and enabled the virtual host
  77 [18:29] <Daviey> puppet knows which package hander to use
  78 [18:29] <Daviey> ie, apt, yum etc
  79 [18:30] <Daviey> Now... if we check to see if apache started.. we'll see it failed... one moment
  80 [18:31] <Daviey> So...
  81 [18:31] <Daviey> (2)No such file or directory: apache2: could not open error log file /var/log/apache/demo.daviey.com-error.log.
  82 [18:31] <Daviey> Unable to open logs
  83 [18:31] <Daviey> This means i made a typo in my template... suggestions on how i should fix this?
  84 [18:31] <Daviey> kim0, Is quite correct with:
  85 [18:31] <Daviey> <kim0> Daviey: should be "apache2" there
  86 [18:31] <Daviey> But... How should i *fix* it?
  87 [18:32] <Daviey> We edit the template of course!
  88 [18:33] <Daviey> Now, we should be able to go to http://demo.daviey.com/
  89 [18:33] <Daviey> (My simple Task didn't try to start apache if it wasn't already running!)
  90 [18:33] <Daviey> notice: /Stage[main]//Node[ip-10-117-82-138]/Apache2::Simple-vhost[demo.daviey.com]/File[/etc/apache2/sites-available/demo.daviey.com]/content: content changed '{md5}5047b9f9a688c04e2697d9fd961960ed' to '{md5}2c32102fd06543c85967276eeee797e2'
  91 [18:34] <Daviey> ^^ Puppet knew it should create a new virtual host, based on the template changing!
  92 [18:34] <Daviey> How neat is that?!
  93 [18:35] <Daviey> Now, in a real life example - puppet would also manage pulling in the website..
  94 [18:35] <Daviey> puppet provides a FileBucket interface..
  95 [18:35] <Daviey> This is similar to rsync, and allows files to be retrieved from there.
  96 [18:35] <Daviey> However, for large files - people often use an external application which is configured via puppet.
  97 [18:36] <Daviey> This could be anything from rsyncd, nfs or event torrent!
  98 [18:36] <Daviey> facter is a really useful tool.  This is where the variables used in the templates are expanded from...  I think of it as lsb_release supercharged.
  99 [18:37] <Daviey> Here is an example of the output, just generated:
 100 [18:37] <Daviey> http://paste.ubuntu.com/584952/
 101 [18:37] <Daviey> This is a list of 'facts' about the system
 102 [18:38] <Daviey> One of the really nice things about the manifests... is that they can be conditional
 103 === Mike is now known as Guest49828
 104 [18:38] <Daviey> So, i could do a different task based on they virtual type (or lack of) for example.
 105 [18:39] <Daviey> There is no point trying to use this machine as a virtual machine server, if it doesn't fit the requirements
 106 [18:39] <Daviey> Usually bare metal - and amount of memory free
 107 [18:39] <Daviey> The configuration files are largely inheritance based, which fully supports overriding of configurations from the base class.
 108 [18:40] <Daviey> When puppet is installed on a client / server basis... it uses SSL for secure communciation between the elements
 109 [18:41] <Daviey> The server runs on port 8140. so make sure firewall is opened (or ec2 security group allows communication!)
 110 [18:41] <Daviey> Client (Agent) - puppetd
 111 [18:41] <Daviey> Server - puppetmasterd
 112 [18:41] <Daviey> ^^ This is the name of the applications
 113 [18:41] <Daviey> The puppetd runs on all the clients, and polls the Server with the default value of every 30 minutes looking for changes
 114 [18:42] <Daviey> It defaults to looking for the dns hostname of 'puppet'
 115 [18:42] <Daviey> So, it's a good idea for the puppet master to have that dns entry set for a local network
 116 [18:42] <Daviey> Equally, i could have set puppet.mydomain.com
 117 [18:43] <Daviey> This is probably a good place to stop the demo.  I will make my puppet configuration avaiable for others to experiment with.
 118 [18:43] <Daviey> It really is not as complicated as it seems to get started.
 119 [18:43] <Daviey> When i first tried puppet, i found the 'getting started' docs to be somewhat complicated.
 120 [18:44] <Daviey> I would recommend people start with a minimal example like this.. and then build from there.
 121 [18:44] <Daviey> The puppet website has some excellent recipies to use as an example... but probably a good idea to start simple.
 122 [18:44] <Daviey> I will now take questions, and answer them as best as i can
 123 === drc is now known as Guest51125
 124 [18:45] <Daviey> Annnnd. classbot, i hate you
 125 [18:45] <Daviey> classbot isn't +v
 126 [18:45] <Daviey> <ClassBot> sveiss asked: how large is 'large'? is there a rule of thumb as to how much data a FileBucket can cope with? -- There is 1 additional question in the queue.
 127 [18:46] <Daviey> sveiss, that is a good question.. I seem to remember reading that since 2.6... massive improvements have gone into increasing it's efficiency
 128 [18:46] <Daviey> However, it is still believed to be the likely bottlekneck
 129 [18:47] <Daviey> I haven't found the later versions to suffer to badly from this bottlekneck
 130 [18:47] <Daviey> but others have commented.
 131 [18:47] <Daviey> I think it depends on load..
 132 [18:48] <Daviey> I would ask that if you do try the filebucket that you report back to the ubuntu server team with your success.
 133 [18:48] <Daviey> (We often don't get enough feedback)
 134 [18:48] <ClassBot> sveiss asked: how large is 'large'? is there a rule of thumb as to how much data a FileBucket can cope with?
 135 [18:49] <Daviey> kim0 asked: Wouldn't clients looking for dns name "puppet" and blindly following it .. be a secruity risk
 136 [18:49] <Daviey> Well yes.. this is true.. This is one of the reasons SSL is used.
 137 [18:49] <Daviey> Essentially, the pupper master (usually has a self signed key)
 138 [18:49] <Daviey> but the client needs to accept it.
 139 [18:49] <Daviey> This would normally happen as part of the installation, or bootstrapping
 140 [18:50] <Daviey> Which is an area before puppet works.
 141 [18:50] <ClassBot> kim0 asked: Wouldn't clients looking for dns name "puppet" and blindly following it .. be a secruity risk
 142 [18:51] <Daviey> Wow.. i now understand ClassBot
 143 [18:51] <ClassBot> There are 10 minutes remaining in the current session.
 144 [18:51] <ClassBot> kim0 asked: Do you reuse ready made recipies
 145 [18:52] <Daviey> You would be foolish not to!
 146 [18:52] <Daviey> There is a true gem of samples on the puppet wiki, and other locations.
 147 [18:52] <Daviey> Additionally, there are additional modules
 148 [18:52] <Daviey> Which allow you to reduce the burden of what you need to do
 149 [18:53] <Daviey> shttp://forge.puppetlabs.com/
 150 [18:53] <Daviey> http://forge.puppetlabs.com/ , rather
 151 [18:53] <Daviey> If there are no more questions, i will end my session.
 152 [18:54] <Daviey> I would like to thank everyone for coming
 153 [18:54] <Daviey> Please do experiment with puppet, and report back to us.
 154 [18:54] <Daviey> We are a friendly team, which hang around in #ubuntu-server
 155 [18:54] <Daviey> Thank you for your time.
 156 [18:55]  * kim0 claps
 157 [18:55] <kim0> Thanks Daviey for the awesome session