== Dev Week -- Automated packaging with pkgme -- james_w -- Thu, 3rd Feb, 2012 == {{{#!irc [20:01] hello everyone, my name is James Westby, and I'm a developer of pkgme [20:01] later you'll be able to play along, if you want to prepare for that then you can run "sudo apt-get install devscripts debhelper python-virtualenv bzr" and then "bzr branch lp:pkgme" [20:01] that should get you set up with the packages you need and get you a copy of the code [20:02] with that out of the way, let's go back to the beginning [20:02] pkgme is a tool to help you package applications for Ubuntu [20:03] after you've written an application you need to package it up so that you can get it in to a PPA and then in to the software-center [20:04] it may be that you know how to do that already, and if so great, but not everyone knows how to do it already [20:04] if you don't know how to do it then you can learn yourself, find someone that does know, or try and use a tool to do it for you [20:05] in Debian/Ubuntu there are three classes of tool that help you to do this [20:05] the first is like checkinstall, which is nicer than installing software without it, but isn't suited for producing packages to distribute to other people [20:05] the second class is things like dh-make [20:06] this gives you a skeleton to work with, but usually you need to know a lot about packaging to get something useful [20:06] so it's mainly used by Debian/Ubuntu developers who want a skeleton to start from, or by those who are reading packaging guides [20:06] but it doesn't really help that last group [20:07] the third class is specialised tools like dh-make-perl [20:07] these deal with one type of package (in that case perl libraries) [20:07] they do it well, but you have to know that they exist [20:08] and also they all work differently, and every time someone wants to write a new one then they have to start almost from scratch, and implement the same things such as writing out the package files [20:08] so where does pkgme fit in to this? [20:09] pkgme sits somewhere between the second and third class [20:09] it gets the benefits of the third class, in that it produces good packages with little knowledge required on the users part [20:10] but it has the advantage that it has a single user interface, it re-uses code across the different types of packages, and you there's only one place to look to know if your package type is supported [20:10] obviously though, it needs to know how to handle each type of package [20:11] here a type of package is referring to a set of packages that have a convention about how they should be packaged [20:11] this means you have things like "python with a setup.py" [20:11] perl with a Makefile.PL [20:11] java with an ant build.xml [20:12] and it extends further as well, we recently added a backend for pdfs, to support the books and magazines that are being sold in Ubuntu Software Center now [20:12] each of these types of package is supported through a pkgme "backend" [20:13] the core of pkgme takes care of all the intracacies of debian package files etc., and the backend supplies the knowledge of that type of package [20:13] so when you run pkgme it first asks each backend to decide whether it can deal with the type of package in question [20:13] each backend returns a score [20:13] they can return 0 if they don't know what it is [20:14] or they can return 10 if they know how to provide some information [20:14] the reason it is is score is so that you can have more specialised backends [20:14] for instance, a ruby backend may see a ruby package and report "10" [20:15] but then a ruby-on-rails backend could see that it was really a RoR project, not a plain ruby one, and take over by reporting 20 for the score [20:15] this isn't limited to two levels either [20:15] there may be a particular subclass of RoRs projects that could have its own backend [20:16] once the backends have all reported their scores then the one with the highest score is used for the next part [20:16] at this point pkgme starts asking the backend for some information about the project [20:16] "what is the name of the project?" [20:16] "what is the version?" [20:16] "what dependencies does it have?" [20:16] "what's the description?" [20:16] etc. [20:17] once pkgme has all this information it puts it in to its templates and writes out the packaging [20:17] so, let's see how all this fits together [20:17] if you have downloaded the branch of pkgme you will see that it has no ./debian/ directory [20:18] this means that it itself is not packaged [20:18] so let's try using pkgme on itself [20:18] cd in to the pkgme directory that bzr gave you [20:18] and run: [20:18] virtualenv --no-site-packages virtualenv [20:18] source ./virtualenv/bin/activate [20:18] python setup.py develop [20:18] pkgme [20:19] it will think for a few seconds, and then will have written out the debian directory [20:19] it will then try and build the source package [20:19] that may or may not work for you (given that it's supposed to be used by the person that built the app) [20:19] that's it, we didn't have to tell pkgme anything about what we were doing, it just figured everything out, and made some sensible decisions for us [20:20] There are 10 minutes remaining in the current session. [20:20] it's likely always going to be possible that an experienced packager will find some better way of packaging the app, but that's ok [20:20] pkgme will produce something workable, which is the immediate goal [20:21] there's one thing I've glossed over so far [20:21] what if pkgme doesn't know how to deal with your particular application? [20:21] there are two possibilities here [20:21] the first is that your app looks like one of the types that pkgme knows about, but differs somehow [20:21] in these cases pkgme will give you an error, or the package won't work [20:22] if it doesn't turn out to be a pkgme bug, then unfortunately it's a sign that you will have to learn something about packaging, or find someone that does [20:22] we will try and accomodate different ways of doing things, but we can't have an automated tool know how to deal with everything [20:23] so when you are writing an app, stick to the conventions of whatever type of project you are writing [20:23] the second case is that pkgme doesn't know how to handle your type of project [20:23] in those cases you would need to write a pkgme backend [20:23] or at least work together with us to write it [20:24] this doesn't need any packaging knowledge (though it doesn't hurt) [20:24] as I said before there are two things the backend needs to be able to do [20:24] the first is decide whether a particular project is something it can handle [20:24] this is usually looking for a particular file (e.g. setup.py) and maybe some other checks [20:24] the second thing is answer some questions about the project [20:24] what the name of it is [20:25] what the dependencies are [20:25] There are 5 minutes remaining in the current session. [20:25] all things that don't really need any packaging knowledge [20:26] so if you are in this situation find us on launchpad (https://launchpad.net/~pkgme-developers) or on IRC (#pkgme) [20:26] and we'll help you write a backend (it takes about an hour to write something useful in my experience) [20:27] then you and everyone who writes the same types of apps can benefit from automatic packaging [20:27] any questions? [20:27] in the meantime I'll write a little about what we at Canonical are building based on pkgme [20:28] when you submit a commercial application to https://developer.ubuntu.com/dev then Canonical will help you package it [20:28] in order to speed that process up and allow more applications to be available on Ubuntu we are putting pkgme behind that form so that an attempt will be made to package your app automatically [20:29] for certain types of application at least [20:29] in addition, we're going to be trying to help libre applications too, by working with the ARB to have pkgme help developers and them to package applications [20:29] tomalan asked: ​ can pkgme als guess dependencies (e.g. by examining PKG_CHECK_MODULE in configure.ac in autotools)? [20:30] that would be how it did it for autotools, yes }}}