Ubuntu Wiki

   1 [17:01] <m_3> hi all
   2 [17:01] <m_3> I'm Mark... and I'm a <borat>CharmingGuy</borat>
   3 [17:01] <m_3> so I'd like to cover:
   4 [17:02] <m_3> - basic intro to juju and service orchestration
   5 [17:02] <m_3> - anatomy of a charm
   6 [17:03] <m_3> rather than concentrate on a single charm, I'll give examples from several different charms for common services that most will be familiar with
   7 [17:04] <m_3> I'll be chatting here, and using a juju-classroom in ec2 to show examples
   8 [17:09] <m_3> argh!
   9 [17:09] <m_3> sorry, conference wifi just cut me off
  10 [17:10] <m_3> ok, so I was saying you can see examples with either a browser or ssh... lemme paste the url:
  11 [17:10] <m_3> ec2-107-21-156-68.compute-1.amazonaws.com
  12 [17:10] <m_3> you can go there in a browser and log in with guest/guest
  13 [17:11] <m_3> or ssh directly there w/ same user/pass
  14 [17:11] <m_3> ok, so juju is a new set of devops tools baked into ubuntu server
  15 [17:11] <m_3> it's a fundamentally new paradigm in devops
  16 [17:11] <m_3> it's not concerned with just configuring machines or automating that process
  17 [17:12] <m_3> it takes it to the next level of looking at your entire infrastructure as a set of services
  18 [17:12] <m_3> services with varied lifecycles that interact with each other
  19 [17:13] <m_3> examples of services... applications, load balancers, databases, monitors, storage, caches, etc
  20 [17:13] <m_3> the list goes on and looks different for different people
  21 [17:14] <m_3> well juju's about maintaining stacks of services within your infrastructure
  22 [17:14] <m_3> take a simple example
  23 [17:15] <m_3> gimme a sec to get reconnected to it
  24 [17:15] <m_3> ok, now in the ec2 instance, you can see a simple stack of services
  25 [17:15] <m_3> this is what the end-user's view of juju will be
  26 [17:16] <m_3> juju bootstrap
  27 [17:16] <m_3> juju deploy [options] service
  28 [17:16] <m_3> juju add-relation service1 service2
  29 [17:16] <m_3> juju expose service
  30 [17:17] <m_3> the last step exposes the service to the outside world (you typically want this to happen ina gated manner)
  31 [17:17] <m_3> that's a stack that deploys a mongo database, a node.js app, and relates the two
  32 [17:18] <m_3> there's some additional stuff commented out that hints at where we can go next with this
  33 [17:18] <m_3> adding extra services and then scaling service units out
  34 [17:18] <m_3> but that's the basic process
  35 [17:18] <m_3> bootstrap, deploy, relate, expose
  36 [17:18] <m_3> so the way juju represents services is through templates we call charms
  37 [17:19] <m_3> there are charms for all sorts of standard services
  38 [17:19] <m_3> mongo, ganglia, mediawiki, mysql, etc etc
  39 [17:19] <m_3> the list is growing every day
  40 [17:20] <m_3> so these charms "distill" the best-practices of the devops peeps that've been working with those services
  41 [17:20] <m_3> some charms are simple package type installations
  42 [17:21] <m_3> some charms are really complex sets of packages, pulls from repos, and templated code
  43 [17:21] <m_3> so these charms "distill" the best-practices of the devops peeps that've been working with those services
  44 [17:21] <m_3> sorry, paste-barf
  45 [17:21] <m_3> the most complex example I can think of at the moment is openstack
  46 [17:22] <m_3> we've got charms built (nod to adam_g) that deploy openstack on bare-metal clusters using juju
  47 [17:22] <m_3> we'll start with simple ones in a bit
  48 [17:23] <m_3> so juju orchestrates services
  49 [17:23] <m_3> represented as charms
  50 [17:23] <m_3> it's essentially a low-level eventing framework as we'll see
  51 [17:24] <m_3> take a look at a more mature stack for a sec (ec2 example)
  52 [17:25] <m_3> this is a more realistic infrastructure than most examples you deal wth
  53 [17:26] <m_3> we're going to delve into the anatomy of charms in some detail, but the big picture is that these charms can be combined into larger stacks of services
  54 [17:26] <m_3> ok, so next is anatomy of a charm
  55 [17:27] <m_3> I'll pause for questions on juju in general before we do that though
  56 [17:27] <m_3> #question
  57 [17:27] <m_3> hmmm... ok, that's not it... I'll check out #ubuntu-classroom-chat
  58 [17:28] <m_3> I got a "this is going to be over our heads"
  59 [17:28] <m_3> and a question "what is juju built on"
  60 [17:29] <m_3> totally not over your heads... wait til you see the charms themselves
  61 [17:29] <m_3> they can be written in any language
  62 [17:29] <m_3> and it's got a really great learning curve
  63 [17:29] <m_3> juju's a basic eventing framework
  64 [17:29] <m_3> uses twisted in the agents
  65 [17:30] <m_3> and uses zookeeper to keep track of it all
  66 [17:31] <m_3> yeah, it's communicating through ssh using std key-injection for most cloud providers
  67 [17:31] <m_3> it runs on ec2, hpcloud, openstack, almost eucalyptus, lxc, bare-metal
  68 [17:32] <m_3> so single-server installs are possible
  69 [17:32] <m_3> just depends on your resources
  70 [17:33] <m_3> an example... I was just developing more on the mongo-node app
  71 [17:33] <m_3> typical production stack for that would be haproxy head, dozen node instances, and a 4-instance mongo replset
  72 [17:33] <m_3> on my local machine, I'll just run a single mongo and a single node instance
  73 [17:34] <m_3> we'll see how the charms scale out ina sec
  74 [17:35] <m_3> ok, so great questions
  75 [17:36] <m_3> let's look at the anatomy of a charm and I think it'll be a little clearer how the events work
  76 [17:36] <m_3> so think about the configuration for a typical service
  77 [17:36] <m_3> there' stuff you gotta do to just get it installed on the box
  78 [17:37] <m_3> sometimes that's super-simple
  79 [17:37] <m_3> (example)
  80 [17:37] <m_3> the 'open-port' is a juju primitive
  81 [17:37] <m_3> sometimes installation isn't so simple
  82 [17:38] <m_3> juju lets you install services on ubuntu service from packages, of course
  83 [17:38] <m_3> but it also lets you easily install using other methods
  84 [17:38] <m_3> like npm
  85 [17:39] <m_3> or gems
  86 [17:39] <m_3> if your app needs a version of django or rails that isn't in the repos
  87 [17:39] <m_3> then you can install it
  88 [17:39] <m_3> of course you can shoot yourself in the foot
  89 [17:40] <m_3> we take pains to insure the shared charms in the charmstore are somewhat safe
  90 [17:40] <m_3> cryptographically verified payloads, peer reviews, tests
  91 [17:41] <m_3> but ultimately, pretty much anything goes in a juju charm
  92 [17:41] <m_3> so a charm for a service is just a handful of hooks
  93 [17:41] <m_3> (see example)
  94 [17:42] <m_3> install, start, stop, upgrade-charm
  95 [17:43] <m_3> the hooks in this example are all shell script
  96 [17:43] <m_3> but they can be written in anything you'd like
  97 [17:43] <m_3> just need an interpreter/runtime on the instance
  98 [17:43] <m_3> heck, you could even write them in clojure
  99 [17:44] <m_3> you just have to make sure the first "install" script bootstraps that executable env
 100 [17:44] <m_3> binary hooks are cool too
 101 [17:44] <m_3> although for the official ones in the charm store, we'd need transparency
 102 [17:45] <m_3> now each service relates to other services
 103 [17:45] <m_3> this is specified in the metadata for the charm
 104 [17:46] <m_3> what sort of services can plug in to this one... is specified in the provides and requires interfaces
 105 [17:46] <m_3> they have to match
 106 [17:46] <m_3> mysql provides a db:mysql interface
 107 [17:47] <m_3> other services such as mediawiki consume this with a requires section
 108 [17:47] <m_3> juju uses these interface definitions (they're loose, you can make up your own) to wire service units together
 109 [17:48] <m_3> and there are hooks to be called when that happens
 110 [17:48] <m_3> so let's go over basic lifecycle...
 111 [17:49] <m_3> juju spins up mysql (juju deploy --repository ~/charms local:mysql)
 112 [17:49] <m_3> juju spins up mediawiki (..)
 113 [17:50] <m_3> for each of these services... juju spins up a new instance, calls the "install" and "start" hooks in the respective charms
 114 [17:50] <ClassBot> There are 10 minutes remaining in the current session.
 115 [17:50] <m_3> next, we 'juju add-relation mysql mediawiki' and then the magic begins
 116 [17:51] <m_3> ok, so it's not magic... it's just juju calling the relation hooks for each of those services
 117 [17:51] <m_3> it also lets these relation hooks talk to each other
 118 [17:51] <m_3> that's the real win IMO
 119 [17:52] <m_3> think of the stuff you have to do to set up a service
 120 [17:52] <m_3> juju separates this into the stuff you have to do for installation
 121 [17:52] <m_3> and the stuff you have to do for relation
 122 [17:52] <m_3> it's typically quite isolated... I'm learning
 123 [17:52] <m_3> installing mediawiki
 124 [17:52] <m_3> installing mysql
 125 [17:53] <m_3> but _relating_ them is just:
 126 [17:53] <m_3> create a new db on mysql
 127 [17:53] <m_3> hand over the creds to access this db to the other service
 128 [17:53] <m_3> that's all they really have to know about each other
 129 [17:53] <m_3> (example)
 130 [17:54] <m_3> ok, that's more detail than we need
 131 [17:54] <m_3> simpler example... pgsql
 132 [17:55] <m_3> there that's it
 133 [17:55] <m_3> create a user/passwd/db
 134 [17:55] <ClassBot> There are 5 minutes remaining in the current session.
 135 [17:55] <m_3> and then "relation-set" is a juju primitive that lets this hook hand the info over to the other end of the relation (mediawiki in this case)
 136 [17:57] <m_3> mediawiki will have corresponding 'relation-get's where it just saves that out to /etc/mediawiki/...
 137 [17:57] <m_3> ok, heading to ubuntu-classroom-chat for questions
 138 [17:59] <m_3> also, if you look at nothing else with juju... totally check out the 'juju debug-hooks' command
 139 [17:59] <m_3> it opens a tmux session on the service unit and spawns a new session for each hook as it execs (with the hook's exec env)
 140 [17:59] <m_3> it's cool!
 141 [18:00] <m_3> so there's juju.ubuntu.com/Charms to see these in detail
 142 [18:00] <m_3> yikes... outta time