App Developer Week -- pkgme: Automating The Packaging Of Your Project -- james_w -- Fri, Apr 15th, 2011
1 [20:01] <james_w> hi everyone 2 [20:01] <james_w> My name is James Westby 3 [20:02] <james_w> I'm here to talk to you about pkgme 4 [20:02] <james_w> who has a cool application that they would love to make available to people on Ubuntu? 5 [20:02] <james_w> who doesn't know how to go about packaging their application? 6 [20:02] <james_w> who knows, but doesn't want to be bothered with that? 7 [20:03] <james_w> if that is you then pkgme may just be the tool for you 8 [20:03] <james_w> pkgme's job is to take a look at your project and generate you some packaging for it that will likely work 9 [20:04] <james_w> it's not going to be packaging that is perfect, but it should at least be enough for you to put your application in an PPA and get people testing it 10 [20:05] <james_w> and you can then improve it as needed to suit your use 11 [20:05] <james_w> so, how does pkgme work? Is it magic? 12 [20:05] <james_w> no, it's not magic, it just has a few principles that it uses to make decisions 13 [20:06] <james_w> firstly, you probably wrote your application using some standard technologies/layout 14 [20:06] <james_w> e.g. you might have used autotools and so have a ./configure file 15 [20:06] <james_w> or you might have used python and so have a setup.py 16 [20:06] <james_w> or it might be a KDE application, a rails application, a cmake application, a vala application etc. 17 [20:07] <james_w> pkgme doesn't care what you used, as long as it has heard of it 18 [20:08] <james_w> pkgme has the concept of a "backend", which is a set of scripts that know how to deal with each of those classes 19 [20:08] <james_w> so the first thing it does is look at your project and decide which class it falls in to 20 [20:08] <james_w> if it is a python application with a setup.py then it will activate the backend that it has for that 21 [20:09] <james_w> if you are unlucky then it either won't be able to tell which class your application falls in to, or it won't have a backend for your class 22 [20:09] <james_w> if it's the former then you can file a bug on pkgme and we will fix it 23 [20:09] <james_w> if it is the latter then you can help by contributing the backend for that class. It's pretty easy, and we'll look in more detail at that later 24 [20:10] <ClassBot> akshatj_ asked: So that means no messing around with /debian, pkgme does the dirty job for you? 25 [20:10] <james_w> yep, that's its aim 26 [20:10] <ClassBot> chadadavis asked: So, others can provide other backends, for say Perl modules ? 27 [20:10] <james_w> exactly, we'll look at how to do this later, it's pretty easy, and the more backends there are the better 28 [20:11] <james_w> you could even write a backend for the exact way that you write projects if you like 29 [20:11] <james_w> but if you stick to something common, like perl, then the aim is to have the backend already there, so you don't have to care about any of this 30 === akshatj_ is now known as akshatj 31 [20:12] <james_w> so once pkgme has decided which backend it will use for your project it goes ahead and writes out the debian/ directory with the needed files 32 [20:12] <james_w> let's take a look at it in action 33 [20:12] <james_w> so, pkgme itself isn't yet packaged, so we can test it out on itself 34 [20:13] <james_w> the first thing you need to do is install python-virtualenv, as we'll need that package 35 [20:13] <james_w> then you should run 36 [20:13] <james_w> bzr branch lp:pkgme 37 [20:13] <james_w> to get the code 38 [20:13] <james_w> once that is done then run 39 [20:13] <james_w> virtualenv virtualenv 40 [20:13] <james_w> (yes that is repeated) 41 [20:14] <james_w> that will create you a python "virtualenv" in a directory of the same name. We'll do this so that we don't interfere with the rest of your system while we are testing this 42 [20:14] <james_w> then run 43 [20:14] <james_w> cd pkgme 44 [20:14] <james_w> and you can look around the code 45 [20:14] <james_w> next run 46 [20:14] <james_w> source ../virtualenv/bin/activate 47 [20:14] <james_w> this activates the virtualenv that you just created 48 [20:14] <james_w> then run 49 [20:15] <james_w> python setup.py install 50 [20:15] <james_w> once that completes the setup is over, and we can start to test pkgme 51 [20:15] <james_w> pkgme works on the project that it finds in the current directory 52 [20:16] <james_w> and we are currently in the directory of pkgme 53 [20:16] <james_w> so it will work on itself here 54 [20:16] <james_w> if you run 55 [20:16] <james_w> PKGME_BACKEND_PATHS=pkgme/backends/ pkgme 56 [20:16] <james_w> it will think for a moment and then finish, and you will find a shiny new "debian" directory that wasn't there before, and it contains the packaging that pkgme just created for itself 57 [20:19] <james_w> is it any good though 58 [20:19] <james_w> ? 59 [20:19] <james_w> to try it out you can run "debuild -b -uc -us" 60 [20:19] <james_w> that will build the package, and here it spits out a .deb file that you could install 61 [20:20] <james_w> at this point you could build a source package and upload it to your PPA for others to try 62 [20:21] <james_w> now, pkgme is quite a young project, so this was a bit of a hassle, but soon it will be packaged and ready to go 63 [20:22] <james_w> and you would have just run 64 [20:22] <james_w> sudo apt-get install pkgme 65 [20:22] <james_w> pkgme 66 [20:22] <james_w> and it would have given you the packaging 67 [20:22] <james_w> we will likely add a mode to put the result in your ppa too 68 [20:22] <james_w> so it would be one command to install pkgme, and then one command to get your shiny new application in to a PPA so that people can use it easily 69 [20:24] <james_w> I think that's pretty cool, I don't know about you 70 [20:24] <james_w> we could even have a button on launchpad to do it all for you, so you would commit your code to bzr, push it to launchpad, and click the button 71 [20:24] <james_w> it doesn't get much easier than that, does it? 72 [20:25] <james_w> now, what happens if it doesn't work, or the packaging doesn't work quite right? 73 [20:26] <james_w> often this will be because you have done something that doesn't quite line up with the standard way of doing things in your class of project 74 [20:26] <james_w> because this is your project you may be able to easily change that 75 [20:26] <james_w> plus there will be a way eventually for you to customise the packaging in certain ways for when you can't really change to match the standards 76 [20:27] <james_w> but we want to learn more about those cases before we decide how to implement that 77 [20:27] <james_w> so give it a try on your own projects and let us know how it goes 78 [20:27] <james_w> there is a #pkgme irc channel that you can join 79 [20:28] <james_w> or there is a mailing list as part of the https://launchpad.net/~pkgme-devs team 80 [20:29] <james_w> so, any questions at this point? 81 [20:32] <ClassBot> chadadavis asked: This reminds me of Launchpad recipes. Are there plans to develop these in parallel. It would be cool if I could use pkgme locally to test things and then let launchpad recipes take care of daily builds. 82 [20:32] <james_w> funny, I worked on Launchpad recipes too :-) 83 [20:33] <james_w> I would like to have them working together, but it's very early days 84 [20:33] <james_w> recipes are great, but they still require someone to create the packaging 85 [20:33] <james_w> having a launchpad feature to run pkgme and put the result in a branch, and then create a recipe to build it every day would be great 86 [20:34] <james_w> I'm pretty sure I can convince the Launchpad developers to implement it if we can show the pkgme works well for a lot of people, so I'd love your help to make sure it does 87 [20:35] <ClassBot> stefano-palazzo asked: is there an example hello world program in Python, with all of the setup.py business set up, that I can download? 88 [20:35] <james_w> I don't know actually 89 [20:35] <james_w> I think if you use quickly it creates one for you 90 [20:36] <james_w> http://packages.python.org/an_example_pypi_project/setuptools.html might be of help 91 [20:37] <ClassBot> akshatj asked: Would it have user input for descriptions? The current automatically generated ones are horrible 92 [20:37] <james_w> you are right, they are horrible :-) 93 [20:37] <james_w> I would like to have the ability to specify them somehow, but we haven't decided how that should work yet 94 [20:37] <james_w> if you have an opinion then please let us know 95 [20:37] <james_w> the mailing list I mentioned at https://launchpad.net/~pkgme-devs would be a great place to do that 96 [20:40] <james_w> so, has anyone tried pkgme on their own projects yet? 97 [20:40] <james_w> if you do 98 [20:40] <james_w> ls pkgme/backends 99 [20:40] <james_w> you will see that there are currently only "python" and "vala" backends implemented 100 [20:41] <james_w> chances are that your needs aren't covered by those two 101 [20:41] <james_w> we would love you to help by contributing a backend 102 [20:41] <james_w> they are easy to write, but it's hard for us to know each language/project class well enough to write them ourselves 103 [20:42] <james_w> you don't need to know anything about packaging to write one, we take care of all those parts :-) 104 [20:42] <james_w> what do you do if you want to write a new one? 105 [20:42] <james_w> take a look at http://pkgme.net/doc/backends/index.html 106 [20:42] <james_w> you can start right now in fact 107 [20:43] <james_w> bzr mkdir pkgme/backends/mybackend 108 [20:43] <james_w> the first thing we write is a "want" script 109 [20:43] <james_w> this is used to decide if your backend can handle a particular project 110 [20:43] <james_w> in this script you take a look at the files in the current working directory and decide if you know what to do with them 111 [20:44] <james_w> so, if you want to do a perl backend, then it will likely do something like 112 [20:44] <james_w> if [ -f "Makefile.PL" ]; then 113 [20:44] <james_w> echo 20 114 [20:44] <james_w> fi 115 [20:45] <james_w> if you want to a a java/Ant backend then you would do something like 116 [20:45] <james_w> if [ -f "build.xml" ]; then 117 [20:45] <james_w> echo 20 118 [20:45] <james_w> fi 119 [20:45] <james_w> I don't really know, whereas you do, which is the point of this backend split in pkgme :-) 120 [20:46] <ClassBot> chadadavis asked: Now, if I want to make a backend, it will have certain dependencies, specific to it. So, will the backends be packaged separately. E.g a Perl backend might require many perl modules that others dont' need to install pkgme. 121 [20:46] <james_w> good question. 122 [20:46] <james_w> it probably makes sense to package them separately, and have a metapackage that pulls in all/most of them by default 123 [20:49] <james_w> right, back to our backend 124 [20:50] <james_w> the way pkgme interacts with a backend is to ask it a bunch of questions, each of them to get one piece of information about the project 125 [20:50] <james_w> what is its name? 126 [20:50] <james_w> what version is it? 127 [20:50] <james_w> what dependencies does it have? 128 [20:50] <james_w> what is its description? 129 [20:50] <james_w> the backend is supposed to make its best attempt to answer, but if it can't answer a certain question then pkgme will try and cope 130 [20:51] <james_w> how does it ask these questions? 131 [20:51] <james_w> take a look at the vala backend 132 [20:51] <ClassBot> There are 10 minutes remaining in the current session. 133 [20:51] <james_w> you will see a bunch of files with names like "package_name" 134 [20:51] <james_w> they will be run by pkgme to get the information 135 [20:52] <james_w> so you can create pkgme/backends/mybackend/package_name and echo "myproject" in there, and pkgme will use that as the name 136 [20:52] <james_w> obviously a backend would usually look at the project files in the current directory to find out the name 137 [20:52] <james_w> see the bottom of http://pkgme.net/doc/backends/index.html for the information that pkgme might ask about 138 [20:53] <james_w> the scripts you should create are named the same 139 [20:53] <james_w> as I'm almost out of time, I'll leave it there 140 [20:53] <james_w> any further questions 141 [20:53] <james_w> after the session if you have questions then there is the #pkgme irc channel 142 [20:54] <james_w> and the mailing list at https://launchpad.net/~pkgme-devs 143 [20:54] <james_w> spend one hour writing a pkgme backend, and never have to care about packaging again! :-) 144 [20:56] <ClassBot> There are 5 minutes remaining in the current session. 145 [20:58] <james_w> thanks everyone! 146 [20:58] <james_w> please help it succeed by testing and giving feedback, and contributing backends 147 [20:58] <james_w> it won't succeed otherwise, and you'll be back to creating packaging yourself for all your applications :-) 148 [20:59] <jcastro> alright, thanks james! 149 [21:00] <james_w> now for some unity!