CharmingJuju
Dev Week -- Charming Juju -- m_3 -- Wed, Feb 1st, 2012
1 [17:01] <m_3> hi all
2 [17:01] <m_3> I'm Mark... and I'm a <borat>CharmingGuy</borat>
3 [17:01] <m_3> so I'd like to cover:
4 [17:02] <m_3> - basic intro to juju and service orchestration
5 [17:02] <m_3> - anatomy of a charm
6 [17:03] <m_3> rather than concentrate on a single charm, I'll give examples from several different charms for common services that most will be familiar with
7 [17:04] <m_3> I'll be chatting here, and using a juju-classroom in ec2 to show examples
8 [17:09] <m_3> argh!
9 [17:09] <m_3> sorry, conference wifi just cut me off
10 [17:10] <m_3> ok, so I was saying you can see examples with either a browser or ssh... lemme paste the url:
11 [17:10] <m_3> ec2-107-21-156-68.compute-1.amazonaws.com
12 [17:10] <m_3> you can go there in a browser and log in with guest/guest
13 [17:11] <m_3> or ssh directly there w/ same user/pass
14 [17:11] <m_3> ok, so juju is a new set of devops tools baked into ubuntu server
15 [17:11] <m_3> it's a fundamentally new paradigm in devops
16 [17:11] <m_3> it's not concerned with just configuring machines or automating that process
17 [17:12] <m_3> it takes it to the next level of looking at your entire infrastructure as a set of services
18 [17:12] <m_3> services with varied lifecycles that interact with each other
19 [17:13] <m_3> examples of services... applications, load balancers, databases, monitors, storage, caches, etc
20 [17:13] <m_3> the list goes on and looks different for different people
21 [17:14] <m_3> well juju's about maintaining stacks of services within your infrastructure
22 [17:14] <m_3> take a simple example
23 [17:15] <m_3> gimme a sec to get reconnected to it
24 [17:15] <m_3> ok, now in the ec2 instance, you can see a simple stack of services
25 [17:15] <m_3> this is what the end-user's view of juju will be
26 [17:16] <m_3> juju bootstrap
27 [17:16] <m_3> juju deploy [options] service
28 [17:16] <m_3> juju add-relation service1 service2
29 [17:16] <m_3> juju expose service
30 [17:17] <m_3> the last step exposes the service to the outside world (you typically want this to happen ina gated manner)
31 [17:17] <m_3> that's a stack that deploys a mongo database, a node.js app, and relates the two
32 [17:18] <m_3> there's some additional stuff commented out that hints at where we can go next with this
33 [17:18] <m_3> adding extra services and then scaling service units out
34 [17:18] <m_3> but that's the basic process
35 [17:18] <m_3> bootstrap, deploy, relate, expose
36 [17:18] <m_3> so the way juju represents services is through templates we call charms
37 [17:19] <m_3> there are charms for all sorts of standard services
38 [17:19] <m_3> mongo, ganglia, mediawiki, mysql, etc etc
39 [17:19] <m_3> the list is growing every day
40 [17:20] <m_3> so these charms "distill" the best-practices of the devops peeps that've been working with those services
41 [17:20] <m_3> some charms are simple package type installations
42 [17:21] <m_3> some charms are really complex sets of packages, pulls from repos, and templated code
43 [17:21] <m_3> so these charms "distill" the best-practices of the devops peeps that've been working with those services
44 [17:21] <m_3> sorry, paste-barf
45 [17:21] <m_3> the most complex example I can think of at the moment is openstack
46 [17:22] <m_3> we've got charms built (nod to adam_g) that deploy openstack on bare-metal clusters using juju
47 [17:22] <m_3> we'll start with simple ones in a bit
48 [17:23] <m_3> so juju orchestrates services
49 [17:23] <m_3> represented as charms
50 [17:23] <m_3> it's essentially a low-level eventing framework as we'll see
51 [17:24] <m_3> take a look at a more mature stack for a sec (ec2 example)
52 [17:25] <m_3> this is a more realistic infrastructure than most examples you deal wth
53 [17:26] <m_3> we're going to delve into the anatomy of charms in some detail, but the big picture is that these charms can be combined into larger stacks of services
54 [17:26] <m_3> ok, so next is anatomy of a charm
55 [17:27] <m_3> I'll pause for questions on juju in general before we do that though
56 [17:27] <m_3> #question
57 [17:27] <m_3> hmmm... ok, that's not it... I'll check out #ubuntu-classroom-chat
58 [17:28] <m_3> I got a "this is going to be over our heads"
59 [17:28] <m_3> and a question "what is juju built on"
60 [17:29] <m_3> totally not over your heads... wait til you see the charms themselves
61 [17:29] <m_3> they can be written in any language
62 [17:29] <m_3> and it's got a really great learning curve
63 [17:29] <m_3> juju's a basic eventing framework
64 [17:29] <m_3> uses twisted in the agents
65 [17:30] <m_3> and uses zookeeper to keep track of it all
66 [17:31] <m_3> yeah, it's communicating through ssh using std key-injection for most cloud providers
67 [17:31] <m_3> it runs on ec2, hpcloud, openstack, almost eucalyptus, lxc, bare-metal
68 [17:32] <m_3> so single-server installs are possible
69 [17:32] <m_3> just depends on your resources
70 [17:33] <m_3> an example... I was just developing more on the mongo-node app
71 [17:33] <m_3> typical production stack for that would be haproxy head, dozen node instances, and a 4-instance mongo replset
72 [17:33] <m_3> on my local machine, I'll just run a single mongo and a single node instance
73 [17:34] <m_3> we'll see how the charms scale out ina sec
74 [17:35] <m_3> ok, so great questions
75 [17:36] <m_3> let's look at the anatomy of a charm and I think it'll be a little clearer how the events work
76 [17:36] <m_3> so think about the configuration for a typical service
77 [17:36] <m_3> there' stuff you gotta do to just get it installed on the box
78 [17:37] <m_3> sometimes that's super-simple
79 [17:37] <m_3> (example)
80 [17:37] <m_3> the 'open-port' is a juju primitive
81 [17:37] <m_3> sometimes installation isn't so simple
82 [17:38] <m_3> juju lets you install services on ubuntu service from packages, of course
83 [17:38] <m_3> but it also lets you easily install using other methods
84 [17:38] <m_3> like npm
85 [17:39] <m_3> or gems
86 [17:39] <m_3> if your app needs a version of django or rails that isn't in the repos
87 [17:39] <m_3> then you can install it
88 [17:39] <m_3> of course you can shoot yourself in the foot
89 [17:40] <m_3> we take pains to insure the shared charms in the charmstore are somewhat safe
90 [17:40] <m_3> cryptographically verified payloads, peer reviews, tests
91 [17:41] <m_3> but ultimately, pretty much anything goes in a juju charm
92 [17:41] <m_3> so a charm for a service is just a handful of hooks
93 [17:41] <m_3> (see example)
94 [17:42] <m_3> install, start, stop, upgrade-charm
95 [17:43] <m_3> the hooks in this example are all shell script
96 [17:43] <m_3> but they can be written in anything you'd like
97 [17:43] <m_3> just need an interpreter/runtime on the instance
98 [17:43] <m_3> heck, you could even write them in clojure
99 [17:44] <m_3> you just have to make sure the first "install" script bootstraps that executable env
100 [17:44] <m_3> binary hooks are cool too
101 [17:44] <m_3> although for the official ones in the charm store, we'd need transparency
102 [17:45] <m_3> now each service relates to other services
103 [17:45] <m_3> this is specified in the metadata for the charm
104 [17:46] <m_3> what sort of services can plug in to this one... is specified in the provides and requires interfaces
105 [17:46] <m_3> they have to match
106 [17:46] <m_3> mysql provides a db:mysql interface
107 [17:47] <m_3> other services such as mediawiki consume this with a requires section
108 [17:47] <m_3> juju uses these interface definitions (they're loose, you can make up your own) to wire service units together
109 [17:48] <m_3> and there are hooks to be called when that happens
110 [17:48] <m_3> so let's go over basic lifecycle...
111 [17:49] <m_3> juju spins up mysql (juju deploy --repository ~/charms local:mysql)
112 [17:49] <m_3> juju spins up mediawiki (..)
113 [17:50] <m_3> for each of these services... juju spins up a new instance, calls the "install" and "start" hooks in the respective charms
114 [17:50] <ClassBot> There are 10 minutes remaining in the current session.
115 [17:50] <m_3> next, we 'juju add-relation mysql mediawiki' and then the magic begins
116 [17:51] <m_3> ok, so it's not magic... it's just juju calling the relation hooks for each of those services
117 [17:51] <m_3> it also lets these relation hooks talk to each other
118 [17:51] <m_3> that's the real win IMO
119 [17:52] <m_3> think of the stuff you have to do to set up a service
120 [17:52] <m_3> juju separates this into the stuff you have to do for installation
121 [17:52] <m_3> and the stuff you have to do for relation
122 [17:52] <m_3> it's typically quite isolated... I'm learning
123 [17:52] <m_3> installing mediawiki
124 [17:52] <m_3> installing mysql
125 [17:53] <m_3> but _relating_ them is just:
126 [17:53] <m_3> create a new db on mysql
127 [17:53] <m_3> hand over the creds to access this db to the other service
128 [17:53] <m_3> that's all they really have to know about each other
129 [17:53] <m_3> (example)
130 [17:54] <m_3> ok, that's more detail than we need
131 [17:54] <m_3> simpler example... pgsql
132 [17:55] <m_3> there that's it
133 [17:55] <m_3> create a user/passwd/db
134 [17:55] <ClassBot> There are 5 minutes remaining in the current session.
135 [17:55] <m_3> and then "relation-set" is a juju primitive that lets this hook hand the info over to the other end of the relation (mediawiki in this case)
136 [17:57] <m_3> mediawiki will have corresponding 'relation-get's where it just saves that out to /etc/mediawiki/...
137 [17:57] <m_3> ok, heading to ubuntu-classroom-chat for questions
138 [17:59] <m_3> also, if you look at nothing else with juju... totally check out the 'juju debug-hooks' command
139 [17:59] <m_3> it opens a tmux session on the service unit and spawns a new session for each hook as it execs (with the hook's exec env)
140 [17:59] <m_3> it's cool!
141 [18:00] <m_3> so there's juju.ubuntu.com/Charms to see these in detail
142 [18:00] <m_3> yikes... outta time
MeetingLogs/devweek1201/CharmingJuju (last edited 2012-02-02 09:25:53 by dholbach)