== Dev Week -- Developing IRC Bots -- tsimpson -- Wed, Mar 2nd, 2011 == {{{#!irc [19:01] Hello everyone \o [19:01] Logs for this session will be available at http://irclogs.ubuntu.com/2011/03/02/%23ubuntu-classroom.html following the conclusion of the session. [19:01] my name is Terence Simpson, and I'm lead developer of the Ubuntu Bots project (that's the one that gives us ubottu and friends) [19:01] I'm also an IRC operator in several Ubuntu channels, and a councillor on the Ubuntu IRC Council [19:01] so I have come to know IRC quite well over the years [19:01] and contrary to some reports, I haven't been hacking on IRC bots since I was 5, I'm just not that 1337 ;) [19:01] this is the Launchpad project for Ubuntu Bots: https://launchpad.net/ubuntu-bots [19:02] today I'm going to talk to you a bit about the IRC protocol, and creating a simple plugin for the supybot IRC bot [19:02] so you may want to 'sudo apt-get install supybot' now if you want to play along [19:02] the IRC bot most of you will be familiar with is ubottu, or one of its ubot* and lubotu* clones [19:03] these bots are all standard supybot IRC bots with our custom plugins, which are developed in the launchpad project above [19:03] later, I'll go through creating a custom plugin, some python knowledge is required for that part [19:03] before that though, you should know a something abou the IRC protocol itself [19:03] some of what I'm going to discuss may not be necessary for developing a supybot plugin, but it's useful to understand what's actually going on [19:04] also, other IRC bots may require more/less knowledge than others, or you may be feeling particularly insane and want to write your own IRC bot/client [19:04] http://www.irchelp.org/irchelp/rfc/ contains lots useful information about the IRC protocol, as well as the RFC (the document describing the IRC protocol) [19:04] the general way IRC works is, you send a command to the IRC server, followed by carriage-return new-line, and it responds in some way [19:04] in this way we say IRC is an "event driven" protocol [19:04] the carriage-return and new-line characters are represented by "\r" and "\n" in character strings, new-line is also known as line-feed [19:05] and the server uses those two characters to determine when you're done sending your commands [19:05] the IRC command you'll care most about when developing an IRC bot, or plugins for IRC bots, is the "PRIVMSG" command [19:05] a PRIVMSG is how I'm talking to you now, it how clients send messages, and despite its name, it's not inherently a PRIVate MeSsaGe. [19:05] when you type a message into your IRC client, and hit enter, the client will send a PRIVMSG command to the server [19:06] command in IRC usually have some arguments/parameters that go along with them [19:06] for PRIVMSG, you'll want to tell the server where you want the message sent to [19:06] and what the message is [19:06] in IRC each argument is separated by a space [19:06] if you need to send an argument with a space in it, it has to be the last argument and it needs to be prefixed with a ':' character [19:06] our messages usually contain spaces, so that's an instance of where you need to do that [19:06] so here's an example [19:07] if I wanted to send the message "Hello World!" to the channel "#mychannel", this is what my client would send: [19:07] PRIVMSG #mychannel :Hello World!\r\n [19:07] the way the server responds to this is to forward that message to all of the other servers it's connected to and clients connected to it [19:07] however, when the server forwards these messages, it adds something called a "prefix" to the message [19:07] his prefix tells everyone who receives the message where it came from, it's made up of your nick name, user/ident, and host. then prefixed with a ':' [19:07] the format is ':nick!ident@host', so the prefix for me is ":tsimpson!~tsimpson@ubuntu/member/stdin" [19:07] when your client sees the "Hello World!" message from me, it would receive this: [19:08] :tsimpson!~tsimpson@ubuntu/member/stdin PRIVMSG #mychannel :Hello World!\r\n [19:08] the only difference between a public channel message and a private one-to-one message is the "target" [19:08] the target is either a channel, as #mychannel is above, or a nick of another user [19:08] there are lots of commands in the IRC protocol, like NICK, JOIN, PART, QUIT, PRIVMSG, NOTICE, MODE, and many numeric commands [19:08] all the numeric commands come from the server and are usually informational or indicate errors [19:08] all these messages have the same form when you receive them [19:08] : : [19:09] where each item after the is optional (depending on the command) [19:09] it's also worth noting that when you get a message from the server, rather than another person, the is the server rather than a user mask [19:09] for instance, if you connect to someserver.freenode.net, that will be the prefix when you get a message from the server. [19:09] using the information in the IRC RFC you could create your own IRC client, and have that do whatever you wanted [19:09] and IRC bot is just an IRC client that reacts to the messages in some custom way [19:09] really, there is no difference between a "real" IRC client, and some script that connects to an IRC server [19:10] another thing to note about IRC, is that it has no character encoding. that is, it's not ASCII, or UTF-8, or anything [19:10] it is up to the client to figure out what each particular message is encoded in [19:10] but most clients are UTF-8 aware by default, and that's what you should use [19:10] IRC is also case insensitive, that is "PRIVMSG #MyChannel" and "privmsg #mychannel" are exactly the same command with exactly the same destination [19:10] the only exception to this is the characters '{', '}', and '|', which are considered the lower-case forms of '[', ']' and '\', respectively [19:11] the reason for that has to do with IRCs Scandinavian origin, and it's something you'll need to be aware of when comparing nick names etc [19:11] from that, you should have a good idea of how ubottu works when responding to factoid requests [19:11] it receives a PRIVMSG command from the server, looks at the message target (a channel or itself) [19:11] if the message is in a channel, it checks that it starts with the '!' character [19:11] then it goes looking in its database for that factoid reply, and sends that off to either the channel or the nick name [19:11] ubottu does more complicated things than that [19:11] like prefixing a nick in the "!factoid | nick" form, and sending the messages in private with the "!factoid > nick" form [19:12] that's a little more complicated, but it all starts with a PRIVMSG command. [19:12] there's lots more I could tell you about the IRC protocol [19:12] but I wanted to give you an example of how to write a plugin for supybot [19:12] some Python knowledge is required here on in, but you should get the general idea even if you don't know Python [19:13] if you've installed supybot, you can see its files in /usr/share/pyshared/supybot/ [19:13] the first thing you need to do, after installing supybot, is to choose a directory where your bot is going to live in [19:13] this will be where all the config file and plugin will go [19:13] ~/bot is a good example, and that's what I'm going to use in this example [19:13] you need to have a terminal open for this, as supybot is set-up via the command-line [19:14] irst you create the directory; "mkdir ~/bot", then change to that directory; "cd ~/bot" [19:14] then you run though the supybot set-up with the "supybot-wizard" command from ~/bot [19:14] you can just hit enter for the first question about bolding, and then 'y' if it works [19:14] choose 'n' for the question "Are you an advanced supybot user?" [19:14] you can look through that later if you want, but it's not necessary right now [19:14] then hit enter to create the directories in the current (~/bot) directory [19:14] for the "IRC network", enter: freenode [19:15] for the server: irc.freenode.net [19:15] then hit enter (or choose 'n') for "Does this connection require connection to a non-standard port?" [19:15] then you need to choose a nick name for your bot, remember that it needs to be unique on the network (so don't choose ubottu ;) [19:15] for this example, I'll refer to the nick "mybot", but you should choose something different [19:15] press enter, or choose 'n', for the question about setting a server password [19:15] next it'll ask if you want the bot to join some channels when it connects, you'll want to choose 'y' here [19:15] you should now get yourself a temporary channel, this is easy enough, you just join an empty channel [19:15] a good idea is to create one based on your nick, so if you are "someone", you'd /join ##someone [19:16] the double-# on freenode indicates it's an "about" or unaffiliated channel, anyone is free to create that kind of channel on freenode [19:16] so, once you're in your new channel, type the name into the supybot wizard and press enter [19:16] supybot comes with many plugins pre-installed, you can look at them later [19:16] for now just answer 'n' when it asks if you want to look at them individually [19:16] next, it will ask if you want to add an owner for the bot, you do [19:17] supybot will only accept certain commands from an owner or an admin [19:17] commands that make it join/part channels or change nicks or quit for example [19:17] type in your IRC nick for the user-name and then pick a password you'll use [19:17] it'll ask twice for the password to make sure you don't mistype [19:17] the next question is about the "prefix char", this is a character that the bot will use to recognise when you are giving it a command [19:17] press 'y', then enter, then you can choose a character. a good choice is the '@' character [19:17] and that's it, a basic set-up for a working supybot IRC bot [19:18] to start the IRC bot and get it connected to IRC, you run the command "supybot mybot.conf" (where "mybot" is the nick you chose for your bot) [19:18] you'll see it connecting from the console, it'll print lots of information there so you can monitor it [19:18] as a note, you can run a supybot instance in "daemon" mode by adding '-d' to the command [19:18] that will make the supybot instance run in the background and log messages to ~/bot/log/messages.log [19:18] for now though, we will want to see the messages in case there are any errors when we start developing a plugin [19:19] which is quite common [19:19] you should see something like "Join to ##someone on freenode synced in 1.15 seconds.", that means the bot has successfully joined the channel you gave it [19:19] and you should see it sitting there in your channel [19:19] the first thing you need to do is let the bot know you are the owner, you do this in /msg as it requires you sending a password [19:19] so you do /msg mybot identify someone my_password [19:19] (changing "someone" and "my_password" for the name of the owner and the password you gave to supybot-wizard) [19:20] you should get a message back with " The operation succeeded." [19:20] the bot knows you are the owner now [19:20] next you should type this in the channel where the bot is: @config supybot.reply.whenNotCommand False [19:20] (replacing '@' with the prefix character you choose) [19:20] I'll come back to why you did that that later [19:20] with its default set-up, the bot won't do very much [19:20] so we are going to make a plugin for the bot to do something :) [19:21] you'll want to open another terminal window/tab and navigate to ~/bot/plugins (cd ~/bot/plugins) [19:21] abhinav19 asked: are there any additional dependencies or requirements for running supybot-wizard as I got some errors ? [19:21] there shouldn't be [19:21] it just requires the basic python stuff [19:22] which should be pre-installed [19:22] ok, so we're in ~/bot/plugins [19:22] hat is where the bot will look for plugins you create for it [19:22] *that [19:22] we are going to create a plugin, called Say, that will make the bot say something when we tell it to [19:22] it will have one command in it, "say", which will repeat everything after the command [19:23] so when we do "@say Hello, World!", our bot will reply with "Hello, World!" [19:23] how exciting :) [19:23] to start we run the command "supybot-plugin-create" [19:23] it'll ask what the name of the plugin should be [19:23] et's choose "Say" [19:23] (plugin names should usually start with an upper-case character) [19:23] the question about if it should be threaded is for more advanced plugins, that may be doing many things at the same time. we don't need that so we choose 'n' [19:23] it'll then ask for your real name, so it can add a copyright and licensing information, go ahead and enter that [19:24] and choose 'y' to use the Supybot license [19:24] you can choose another license for your plugins if you wish, but the supybot license is an open-source license [19:24] the template for the plugin should now be in the "Say" sub-directory at ~/bot/plugins/Say, so navigate there [19:24] you'll see 5 files and one directory, "config.py", "__init__.py", "plugin.py", "README.txt", and "test.py" are the files [19:24] the directory is there in case you want to create custom python modules [19:24] we won't be using it in this example, but it does no harm being there [19:24] "config.py" is where all the configuration for the plugin will go, we don't need any configuration for this plugin, so we won't modify that [19:25] "__init__.py" is there to make that directory a python module, you can put some things in there if you wanted, but we don't need to do anything special here [19:25] "plugin.py" is where our actual plugin code will be, we'll come back to that in a second [19:25] "README.txt" is pretty self-explanatory, it's where you'll put some information about your plugin, like a description and any set-up information [19:25] "test.py" is for testing, for complex plugins you can use that with the supybot-test script to run tests on the plugin [19:25] now go ahead and open plugin.py in your favourite editor or python IDE [19:26] the comment block at the top is the your copyright and the supybot license [19:26] then it has some default imports followed by our plugin class "class Say(callbacks.Plugin):" [19:26] all supybot plugin classes need to derive from supybot.callbacks.Plugin in some way, and there are a few special sub-classes too [19:26] then is the line "Class = Say" [19:26] when supybot imports a plugin module, it looks for "Class" in that module to identify the plugin [19:26] and in __init__.py it imports "plugin", and sets Class = plugin.Class [19:27] now we should change the doc-string to something meaningful [19:27] supybot uses the doc-strings of classes and methods to provide the "@help" command [19:27] so it's good practice to always document as you go [19:27] something like """A plugin to say something when told to""" is good enough for this plugin [19:27] when supybot receives a command from the IRC server, it looks for a corresponding method in the plugins it has loaded [19:27] the command is lower-cased and capitalized, so "PRIVMSG" becomes "Privmsg" [19:28] the method supybot looks for is "do" followed by the command [19:28] so, if we want to listen for channel/private message, we need to create an "doPrivmsg" method [19:28] for all of these kinds of method, supybot passes 2 arguments [19:28] an instance of the supybot.irclib.Irc class (or a wrapper class), which represents the IRC client/bot, remember there's really no difference [19:28] and an instance of the supybot.ircmsgs.IrcMsg class, which represents and IRC message [19:28] in our method the Irc class we get will be a 'supybot.callbacks.ReplyIrcProxy' [19:29] it just adds some convenience methods for replying to messages [19:29] the Irc instance will contain information such as the network connected to [19:29] nick name of the bot on that network [19:29] and state information, like what channels are joined and what users are in those channels etc [19:29] as I mentioned earlier, when you want to compare channels or nicks in IRC, there are some special characters to consider [19:29] supybot has some utility function that help us, they are located in the supybot.ircutils module, which is already imported as "ircutils" [19:30] back to the plugin [19:30] in you plugin class (Say), remove the line "pass" [19:30] then create a method called "doPrivmsg" taking 3 arguments, "self", "irc", and "msg" [19:30] you should now have something similar to http://people.ubuntu.com/~tsimpson/botdev/plugin.1.py [19:30] still nothing useful there, but we're about to change that [19:30] this doPrivmsg method of our plugin class will be called when ever the bot sees a message [19:31] now, we need some way to tell if the message was addressed to the bot or not [19:31] we don't want our bot to respond to random messages after all [19:31] our bot is addressed for example, if the message starts with the bots nick or command character you set [19:31] now we could write some code to do all that [19:31] but I'm lazy ;) [19:32] and besides, we don't need to [19:32] there is a "addressed" function in the 'supybot.callbacks' module which will do all that for us [19:32] and supybot.callbacks is already imported as 'callbacks' for us [19:32] we give it the bots nick, and the IRC message ('msg' parameter) [19:32] it will either return an empty string, in which case the bot wasn't addressed [19:32] or it will return the message without the part of the message used to address the bot [19:32] for example, if the message way "mybot: this is a message", and we executed this code: [19:32] message = callbacks.addressed(irc.nick, msg) [19:32] the result in 'message' would be "this is a message" [19:33] f the message was just "this is a message", then 'message' would be an empty string '' [19:33] by the way, as I mentioned above the 'irc' parameter contains information, such as the bots nick. that's what 'irc.nick' gives us [19:33] in the 'msg' parameter there is the property 'args' [19:33] that holds a tuple with all the arguments of the message [19:33] remember that PRIVMSG has 2 arguments [19:33] the "target" (a channel or nick), and the message [19:33] the target is in 'msg.args[0]' and the message is in 'msg.args[1]' [19:33] when we call 'callbacks.addressed', it looks at both msg.args[0] and msg.args[1] [19:34] it will check if msg.args[0] (the target of the message) is the bots nick, in which case it is a private message [19:34] so, the first thing we do in our doPrivmsg method, is this: [19:34] message = callbacks.addressed(irc.nick, msg) [19:34] then we can check if 'message' is empty and, in that case, just return [19:34] this is the code: [19:34] if not message: [19:34] return [19:34] that way our plugin will just ignore all the messages which are not addressed to it [19:34] next we will need to check that the message is our "say" command [19:35] we can do this by splitting off the first word of 'message' and checking that it equals 'say' [19:35] we'll also want to remove it from the 'message' string, as it's not part of what we are supposed to repeat [19:35] this is the code: [19:35] bot_cmd = message.split(None, 1)[0] [19:35] message = message[len(bot_cmd):].strip() [19:35] here we're also slicing the 'message' to remove the command, and calling strip on that [19:35] the command could well just be one word, so we can't just split it and assign to two variables [19:35] we need to use strip() so that we don't have any leading space if/when we reply [19:35] now we check if the command in 'bot_cmd' is the one we are looking for [19:36] this is the code: [19:36] if bot_cmd.lower() != 'say': [19:36] return [19:36] we lower-case the command and check if it's "say", if not we just return as we aren't interested in other commands [19:36] the next thing we need to do is check if we were actually given a message to repeat === mrjazzcat is now known as mrjazzcat-lunch [19:36] there's not much point calling a command that repeats a message without a message to repeat! [19:36] if we weren't give a message, then we need to respond with an error message indicating that we need more arguments [19:36] remember that 'message' is now whatever was left after we removed the command [19:36] this is the code: [19:36] if not message: [19:36] irc.error("I need something to say") [19:36] return [19:37] 'error' is a method on the 'irc' object that lets us respond with an error message [19:37] it will prefix "Error: " to whatever message you pass to it [19:37] if we get past that part of the code, then we know three things [19:37] 1) we know the bot was addressed in some way [19:37] 2) we know the command is "say" [19:37] 3) we know we have the message to repeat [19:37] the only thing left to do, is actually repeat the message! [19:37] this is the code: [19:37] irc.reply(message) [19:37] that's it, the 'reply' method does exactly what you think it does, replies to the message we got [19:38] now our plugin is ready to be used, it should look something like this: http://people.ubuntu.com/~tsimpson/botdev/plugin.2.py [19:38] that's around 12 lines of python we've added, and it does exactly what we want :) [19:38] to tell you bot to load that plugin, you issue the command "@load Say" in the channel it's in, or in a /msg [19:38] you should see a "The operation succeeded." message from the bot [19:38] if you get an error message, you probably have some typo in the code [19:38] now you can test the plugin out [19:39] I'll have nubotu join here to show you how it should work [19:39] nubotu is my general testing/development bot [19:39] @load Say [19:39] tsimpson: The operation succeeded. [19:39] @say Hello, World! [19:39] tsimpson: Hello, World! [19:39] the bot won't respond to this message, callbacks.addressed() will return an empty string [19:39] @the bot won't respond to this either, as "the" is not the "say" command we want [19:39] nubotu: and it won't respond here, as "and" isn't the "say" command either [19:40] this will cause nubotu to respond with an error, as I'm leaving out the message [19:40] @say [19:40] tsimpson: Error: I need something to say [19:40] nifty ey? [19:40] now, even though that's only around 12 lines of code [19:40] it's still an awful lot of code for a simple repeater command [19:40] fortunately supybot can help us some more here [19:40] in the above example, we didn't really create a command [19:40] at least not in the sense supybot cares about [19:40] remember we did "@config supybot.reply.whenNotCommand False"? [19:41] if we set that back to True, you'll see what I mean [19:41] @config supybot.reply.whenNotCommand True [19:41] tsimpson: The operation succeeded. [19:41] @say [19:41] tsimpson: Error: "say" is not a valid command. [19:41] tsimpson: Error: I need something to say [19:41] @say Hello, World! [19:41] tsimpson: Error: The "Say" plugin is loaded, but there is no command named "Hello," in it. Try "list Say" to see the commands in the "Say" plugin. [19:41] tsimpson: Hello, World! [19:41] that's not so good :( [19:41] in supybot, we create command by defining a method of the same name, and then "wrapping" it [19:41] I'll talk more about wrap in a second [19:41] so now go ahead and delete the entire "doPrivmsg" method from your plugin.py [19:42] the whole thing [19:42] we don't need it [19:42] and replace it with this: [19:42] def say(self, irc, msg, args, message): [19:42] """ [19:42] Repeats [19:42] """ [19:42] irc.reply(message) [19:42] and that's our 'say' command [19:42] the doc-string describes how to use the command, we take only one argument '', so that goes on the first line [19:42] then we have a blank line, followed by a description of the command. [19:42] this will be shown when we do @help say [19:42] the irc and msg parameters are pretty much the same as in our previous doPrivmsg method [19:43] except that 'irc' will usually be a supybot.callbacks.NestedCommandsIrcProxy, that' not something you really need to care about though [19:43] you don't need to worry about the 'args' parameter for now, it's for more advanced commands [19:43] the 'message' parameter is what is going to hold the message we are supposed to say, we don't need to do anything special like with doPrivmsg [19:43] when our command method is called, we *will* have a message to repeat, 'wrap' will make sure for us [19:43] next we need to tell supybot that that method is a command, we do this using the 'wrap' function, which is imported from supybot.commands [19:43] put this under you method: [19:43] at the same indentation as "def" [19:44] say = wrap(say, ['text']) [19:44] the 'wrap' function takes a method, and then a list of so-called "converters" [19:44] in this case, we only want some text, we don't care what it is as long as it's there [19:44] by the way, if you are creating a command that doesn't take any parameters, you can use wrap as a method decorator by putting '@wrap' before the definition [19:44] but you'll usually want to have some parameters for your commands [19:44] now your plugin.py should look like this: http://people.ubuntu.com/~tsimpson/botdev/plugin.3.py [19:44] without all the comments and doc-strings, it's just 3 lines. nice :) [19:45] let's reload the plugin so the changes take effect [19:45] @reload Say [19:45] tsimpson: The operation succeeded. [19:45] now we test [19:45] @help say [19:45] tsimpson: (say ) -- Repeats [19:45] @say Hello, World! [19:45] tsimpson: Hello, World! [19:45] @say [19:45] tsimpson: (say ) -- Repeats [19:45] nubotu: say hello to all the people [19:45] tsimpson: hello to all the people [19:45] it works! [19:45] by the way, if you do '@config supybot.reply.withNickPrefix False', it won't prefix your nick to the response [19:46] @config supybot.reply.withNickPrefix False [19:46] tsimpson: The operation succeeded. [19:46] nubotu: say hello to all the nice people [19:46] hello to all the nice people [19:46] and that's the _basics_ of building plugins for supybot [19:46] things can get a lot more complicated very quickly, have a look at our plugins at: http://bazaar.launchpad.net/~ubuntu-bots/ubuntu-bots/devel/files [19:46] that's basically ubottu right there [19:46] there's also some useful documentation for using supybot at http://supybook.fealdia.org/latest/ and http://ubottu.com/supydocs/ [19:46] and the documentation that comes with supybot in /usr/share/doc/supybot/, especially the USING_WRAP.gz document [19:47] oh, and come join us in #ubuntu-bots-devel if you want to help make ubottu even better :) [19:47] I'll answer any questions you may have now [19:47] (but please don't expect me to know the details of other IRC bot programs, I'll do my best though ;) [19:47] nubotu: part [19:48] hey, I just zoomed through the IRC protocol and bot plugin development in 47 minutes [19:49] no one has questions? [19:51] There are 10 minutes remaining in the current session. [19:52] Mkaysi asked: Doesn't Supybot require pysqlite ? [19:52] no [19:52] not the basic supybot [19:52] but as the plugins are just python modules [19:53] they can, and do, have their own dependencies [19:53] Encyclopedia (the factoid plugin for ubottu), does use it, for example [19:54] we plan to change that though :) [19:55] I should probably end by saying that supybot isn't the only IRC bot out there [19:55] there are many others [19:55] in different languages [19:55] and with different capabilities [19:56] There are 5 minutes remaining in the current session. [19:56] we use supybot mostly because it's the easiest to develop in, as Ubuntu comes with python anyway [19:56] but other bots can be, and are, just as good or better for you [19:56] Mkaysi asked: What things I should tell supybot to forget from Encyclopedia? In example op ops calltheops owner. [19:57] there is detailed use info on the bot wiki [19:57] http://ubottu.com/devel/wiki [19:57] and we can help in #ubuntu-bots too [19:57] chadadavis asked: Can a I configure a plugin to load on start (myplugin.conf)? [19:58] by default supybot loads all the plugins that were running when the bot shut down [19:58] so unless you @unload a plugin, it will auto-load [19:59] if you have any more question, the bots team hangs out in #ubuntu-bots and #ubuntu-bots-devel [19:59] we'll help you there :) [20:00] thanks for listening, and I hope you'll all come help us make ubottu rock! :D }}}