== Dev Week -- Charming Juju -- m_3 -- Wed, Feb 1st, 2012 == {{{#!irc [17:01] hi all [17:01] I'm Mark... and I'm a CharmingGuy [17:01] so I'd like to cover: [17:02] - basic intro to juju and service orchestration [17:02] - anatomy of a charm [17:03] rather than concentrate on a single charm, I'll give examples from several different charms for common services that most will be familiar with [17:04] I'll be chatting here, and using a juju-classroom in ec2 to show examples [17:09] argh! [17:09] sorry, conference wifi just cut me off [17:10] ok, so I was saying you can see examples with either a browser or ssh... lemme paste the url: [17:10] ec2-107-21-156-68.compute-1.amazonaws.com [17:10] you can go there in a browser and log in with guest/guest [17:11] or ssh directly there w/ same user/pass [17:11] ok, so juju is a new set of devops tools baked into ubuntu server [17:11] it's a fundamentally new paradigm in devops [17:11] it's not concerned with just configuring machines or automating that process [17:12] it takes it to the next level of looking at your entire infrastructure as a set of services [17:12] services with varied lifecycles that interact with each other [17:13] examples of services... applications, load balancers, databases, monitors, storage, caches, etc [17:13] the list goes on and looks different for different people [17:14] well juju's about maintaining stacks of services within your infrastructure [17:14] take a simple example [17:15] gimme a sec to get reconnected to it [17:15] ok, now in the ec2 instance, you can see a simple stack of services [17:15] this is what the end-user's view of juju will be [17:16] juju bootstrap [17:16] juju deploy [options] service [17:16] juju add-relation service1 service2 [17:16] juju expose service [17:17] the last step exposes the service to the outside world (you typically want this to happen ina gated manner) [17:17] that's a stack that deploys a mongo database, a node.js app, and relates the two [17:18] there's some additional stuff commented out that hints at where we can go next with this [17:18] adding extra services and then scaling service units out [17:18] but that's the basic process [17:18] bootstrap, deploy, relate, expose [17:18] so the way juju represents services is through templates we call charms [17:19] there are charms for all sorts of standard services [17:19] mongo, ganglia, mediawiki, mysql, etc etc [17:19] the list is growing every day [17:20] so these charms "distill" the best-practices of the devops peeps that've been working with those services [17:20] some charms are simple package type installations [17:21] some charms are really complex sets of packages, pulls from repos, and templated code [17:21] so these charms "distill" the best-practices of the devops peeps that've been working with those services [17:21] sorry, paste-barf [17:21] the most complex example I can think of at the moment is openstack [17:22] we've got charms built (nod to adam_g) that deploy openstack on bare-metal clusters using juju [17:22] we'll start with simple ones in a bit [17:23] so juju orchestrates services [17:23] represented as charms [17:23] it's essentially a low-level eventing framework as we'll see [17:24] take a look at a more mature stack for a sec (ec2 example) [17:25] this is a more realistic infrastructure than most examples you deal wth [17:26] we're going to delve into the anatomy of charms in some detail, but the big picture is that these charms can be combined into larger stacks of services [17:26] ok, so next is anatomy of a charm [17:27] I'll pause for questions on juju in general before we do that though [17:27] #question [17:27] hmmm... ok, that's not it... I'll check out #ubuntu-classroom-chat [17:28] I got a "this is going to be over our heads" [17:28] and a question "what is juju built on" [17:29] totally not over your heads... wait til you see the charms themselves [17:29] they can be written in any language [17:29] and it's got a really great learning curve [17:29] juju's a basic eventing framework [17:29] uses twisted in the agents [17:30] and uses zookeeper to keep track of it all [17:31] yeah, it's communicating through ssh using std key-injection for most cloud providers [17:31] it runs on ec2, hpcloud, openstack, almost eucalyptus, lxc, bare-metal [17:32] so single-server installs are possible [17:32] just depends on your resources [17:33] an example... I was just developing more on the mongo-node app [17:33] typical production stack for that would be haproxy head, dozen node instances, and a 4-instance mongo replset [17:33] on my local machine, I'll just run a single mongo and a single node instance [17:34] we'll see how the charms scale out ina sec [17:35] ok, so great questions [17:36] let's look at the anatomy of a charm and I think it'll be a little clearer how the events work [17:36] so think about the configuration for a typical service [17:36] there' stuff you gotta do to just get it installed on the box [17:37] sometimes that's super-simple [17:37] (example) [17:37] the 'open-port' is a juju primitive [17:37] sometimes installation isn't so simple [17:38] juju lets you install services on ubuntu service from packages, of course [17:38] but it also lets you easily install using other methods [17:38] like npm [17:39] or gems [17:39] if your app needs a version of django or rails that isn't in the repos [17:39] then you can install it [17:39] of course you can shoot yourself in the foot [17:40] we take pains to insure the shared charms in the charmstore are somewhat safe [17:40] cryptographically verified payloads, peer reviews, tests [17:41] but ultimately, pretty much anything goes in a juju charm [17:41] so a charm for a service is just a handful of hooks [17:41] (see example) [17:42] install, start, stop, upgrade-charm [17:43] the hooks in this example are all shell script [17:43] but they can be written in anything you'd like [17:43] just need an interpreter/runtime on the instance [17:43] heck, you could even write them in clojure [17:44] you just have to make sure the first "install" script bootstraps that executable env [17:44] binary hooks are cool too [17:44] although for the official ones in the charm store, we'd need transparency [17:45] now each service relates to other services [17:45] this is specified in the metadata for the charm [17:46] what sort of services can plug in to this one... is specified in the provides and requires interfaces [17:46] they have to match [17:46] mysql provides a db:mysql interface [17:47] other services such as mediawiki consume this with a requires section [17:47] juju uses these interface definitions (they're loose, you can make up your own) to wire service units together [17:48] and there are hooks to be called when that happens [17:48] so let's go over basic lifecycle... [17:49] juju spins up mysql (juju deploy --repository ~/charms local:mysql) [17:49] juju spins up mediawiki (..) [17:50] for each of these services... juju spins up a new instance, calls the "install" and "start" hooks in the respective charms [17:50] There are 10 minutes remaining in the current session. [17:50] next, we 'juju add-relation mysql mediawiki' and then the magic begins [17:51] ok, so it's not magic... it's just juju calling the relation hooks for each of those services [17:51] it also lets these relation hooks talk to each other [17:51] that's the real win IMO [17:52] think of the stuff you have to do to set up a service [17:52] juju separates this into the stuff you have to do for installation [17:52] and the stuff you have to do for relation [17:52] it's typically quite isolated... I'm learning [17:52] installing mediawiki [17:52] installing mysql [17:53] but _relating_ them is just: [17:53] create a new db on mysql [17:53] hand over the creds to access this db to the other service [17:53] that's all they really have to know about each other [17:53] (example) [17:54] ok, that's more detail than we need [17:54] simpler example... pgsql [17:55] there that's it [17:55] create a user/passwd/db [17:55] There are 5 minutes remaining in the current session. [17:55] and then "relation-set" is a juju primitive that lets this hook hand the info over to the other end of the relation (mediawiki in this case) [17:57] mediawiki will have corresponding 'relation-get's where it just saves that out to /etc/mediawiki/... [17:57] ok, heading to ubuntu-classroom-chat for questions [17:59] also, if you look at nothing else with juju... totally check out the 'juju debug-hooks' command [17:59] it opens a tmux session on the service unit and spawns a new session for each hook as it execs (with the hook's exec env) [17:59] it's cool! [18:00] so there's juju.ubuntu.com/Charms to see these in detail [18:00] yikes... outta time }}}