Ubuntu Wiki

Dev Week -- Packaging from scratch -- Laney -- Fri Sep 4th, 2009

UTC

(03:01:55 PM) Laney: Alright everyone, let's get started!
(03:02:04 PM) Laney: Who's here? Say hi in #ubuntu-classroom-chat
(03:03:26 PM) Laney: Yay, looks healthy
(03:03:39 PM) Laney: (sorry if I go silent for a bit... connectivity problems)
(03:03:58 PM) Laney: So... we're here to learn how to package from scratch
(03:04:13 PM) Laney: take the upstream source tarball and end up with a .deb that users can install on their systems
(03:04:37 PM) Laney: ...and if you persevere enough, install using apt
(03:05:11 PM) Laney: Let's get started, as time is already ticking away
(03:05:30 PM) Laney: Earlier in the week I perused the needs-packaging bugs that have been filed on Launchpad looking for something fun for us to work on in this session
(03:05:48 PM) Laney: I've decided that we should package a little tool for working with PDF files called pdfchain.
(03:06:02 PM) Laney: You can read more about it here: http://pdfchain.sourceforge.net/
(03:06:20 PM) Laney: This is a nice and simple application to package, but one which has a couple of fun twists along the wya
(03:06:44 PM) Laney: So, without further ado, let's download the tarball
(03:06:46 PM) Laney: please run:
(03:06:47 PM) Laney:   wget
(03:06:48 PM) Laney: http://downloads.sourceforge.net/project/pdfchain/pdfchain-0.123/PDF%20Chain%20version%200.123/pdfchain-0.123.tar.gz
(03:06:54 PM) Laney: (on one line)
(03:07:10 PM) AntoineLeclair: shadeslayer: QUESTION:Who decides what to package?
(03:07:22 PM) Laney: shadeslayer: Good question
(03:07:53 PM) Laney: shadeslayer: Individuals do. We have a procedure for requesting packages on Launchpad, but nobody can force you to do the work
(03:08:14 PM) Laney: basically if you want to package an application that nobody else is working on, go ahead and do it :)
(03:09:10 PM) Laney: all got the tarball? We need to move it to the name that the packaging system expects
(03:09:19 PM) Laney: the format is UPSTREAMNAME_VERSION.orig.tar.gz
(03:09:33 PM) Laney: so please mv pdfchain-0.123.tar.gz pdfchain_0.123.orig.tar.gz
(03:09:43 PM) Laney: and then unpack it: tar xzvf pdfchain_0.123.orig.tar.gz
(03:10:02 PM) AntoineLeclair: shadeslayer: QUESTION:if you package an app and it gets uploaded to ubuntu repos,do you have to mange it or does MOTU take care of it?
(03:11:20 PM) Laney: shadeslayer: It will be team maintained in the usual case. Anyone can work on it but that can often mean nobody works on it, so it is expected that once you get a package uploaded you continue to care for it
(03:11:28 PM) Laney: that means managing bugs and keeping track of upstream
(03:11:38 PM) Laney: we don't want unmaintained packages in the archive
(03:11:45 PM) Laney: have we got the tarball unpacked?
(03:12:02 PM) Laney: please change into the directory
(03:12:05 PM) Laney: cd pdfchain-0.123
(03:12:20 PM) Laney: now we need to make a directory to hold all of our packaging data
(03:12:21 PM) Laney: mkdir debian
(03:12:38 PM) Laney: this is where all of the information used to build the package is going to go
(03:13:20 PM) Laney: Let's make empty copies of some of the files we are going to work with
(03:13:39 PM) Laney: I'll explain what these are as we go along
(03:13:40 PM) Laney: touch debian/copyright debian/compat debian/control debian/rules
(03:14:17 PM) Laney: (there is a tool called dh_make to make templates for these files but we won't use it here)
(03:14:56 PM) Laney: The first file we'll work with is the changelog
(03:15:13 PM) Laney: this is used by various pieces of archive software, and is the log of your package's history
(03:15:16 PM) Laney: dch --create --newversion 0.123-0ubuntu1 --package pdfchain --distribution karmic
(03:15:19 PM) Laney: so please run:
(03:15:41 PM) AntoineLeclair: mac_v: <Question> any reason , why the template is not used?
(03:15:42 PM) Laney: "dch" is a tool for managing debian changelog files
(03:16:02 PM) Laney: mac_v: Partly for educational purposes, partly because I don't think it's really necessary
(03:16:11 PM) Laney: dh_make creates a lot of files we don't need here
(03:16:23 PM) Laney: part of what I want to teach you is that packaging is quite easy
(03:16:52 PM) Laney: back to dch -- with this command we've told it to create a new debian changelog file, with the version/package/distribution given
(03:16:59 PM) AntoineLeclair: funkyHat: Question: why are we giving the version an 0ubuntu1 suffix?
(03:17:05 PM) Laney: funkyHat: coming to that )
(03:17:06 PM) Laney: :)
(03:17:45 PM) Laney: A note on the version convention - 0.123 is the upstream version number, which I hope is obvious
(03:18:02 PM) Laney: the - is a separator between the upstream and "debian" (/ubuntu) revision
(03:18:18 PM) Laney: 0 is the revision of the package in Debian itself
(03:18:27 PM) Laney: 0 as it hasn't been uploaded there (I hope)
(03:18:42 PM) Laney: if we were packaging for Debian we would use the version 0.123-1
(03:18:51 PM) Laney: "ubuntu1" means that this is the first revision of the package in Ubuntu
(03:19:15 PM) Laney: so dch should have opened a text editor for you
(03:19:18 PM) Laney: is that right?
(03:20:04 PM) Laney: So now we need to make one small change to the file
(03:20:31 PM) Laney: we need to ensure that when the package is uploaded, the bug that was filed to request the packaging is closed
(03:21:23 PM) Laney: you can see that bug here: https://bugs.launchpad.net/ubuntu/+bug/407982
(03:21:29 PM) Laney: So please change the Closes: #xxxx to LP: #xxxx
(03:21:51 PM) Laney: This instructs the launchpad archive management software to set this bug to "fix released" when the package is uploaded
(03:22:03 PM) Laney: please save and quit the file now
(03:22:06 PM) AntoineLeclair: elopio: Question: if there was no open bug for the package, should we open one before?
(03:22:20 PM) Laney: elopio: It's a good idea to prevent two people doing the same work
(03:22:24 PM) Laney: but it's not mandatory
(03:22:38 PM) Laney: OK the next file we're going to fill in is debian/compat
(03:22:47 PM) Laney: we will be working with Debhelper version 7 so please:
(03:22:49 PM) Laney: echo 7 > debian/compat
(03:22:59 PM) AntoineLeclair: ScottTesterman: QUESTION: If the "Closes" stays, but "LP" is still added before the bug number, will Launchpad still close the bug, or does the word Closes throw it off?
(03:23:07 PM) Laney: this instructs debhelper to use compatibilty level 7
(03:23:28 PM) Laney: see man debhelper for what the various choices are
(03:23:35 PM) Laney: and for more information on what this means
(03:24:01 PM) Laney: ScottTesterman: It will probably break the parser, use LP: #407982
(03:24:28 PM) Laney: OK that file was easy
(03:24:48 PM) Laney: so now we'll move on to the rules file
(03:25:00 PM) Laney: this is the file which describes how to build your package
(03:25:21 PM) Laney: for technical details see http://www.debian.org/doc/debian-policy/ch-source.html
(03:25:30 PM) Laney: for now, please:
(03:25:41 PM) Laney: cp /usr/share/doc/debhelper/examples/rules.tiny debian/rules
(03:25:56 PM) Laney: now open this up in your favourite editor
(03:26:23 PM) Laney: See how simple this is? :)
(03:26:40 PM) Laney: Not so long ago such a short rules file wouldn't have been possible
(03:27:02 PM) Laney: but the magical Joey Hess recently released Debhelper version 7 which allows us to use such short files
(03:27:24 PM) Laney: we now only need to express situations in which the packaging differs from the default behaviour
(03:28:19 PM) Laney: For now we don't know what's going to differ so please quit your editor
(03:28:34 PM) Laney: (we will return to rules later when the package doesn't quite build as expected)
(03:29:02 PM) Laney: now we'll move onto debian/control
(03:29:25 PM) Laney: This is a file which expresses a lot of important metadata about your package
(03:29:46 PM) Laney: Please visit http://pastebin.com/f59a78dd and copy the contents to debian/control
(03:30:04 PM) Laney: I'll quickly explain what the fields mean
(03:30:10 PM) Laney: (speeding up, time is ticking away)
(03:30:16 PM) Laney: Source: name of the source package
(03:30:41 PM) Laney: Section, name of the archive section - allows users to navigate packages by category
(03:30:46 PM) Laney: eg on http://packages.ubuntu.com
(03:30:46 PM) AntoineLeclair: mruiz: QUESTION: What is the default behaviour
(03:30:58 PM) Laney: mruiz: I don't understand, please clarify
(03:31:12 PM) AntoineLeclair: mruiz: QUESTION: What is the default behavior with debhelper 7 ?
(03:31:29 PM) Laney: Priority: how important it is that the user installs the package
(03:31:57 PM) Laney: mruiz: I don't have time to explain, but basically ./configure && make && install (or the appropriate based on the build system in use)
(03:32:12 PM) AntoineLeclair: mruiz: QUESTION: What is the default behavior with debhelper 7 (because debian/rules seems to be black magic)? ;-)
(03:32:34 PM) Laney: Maintainer: Who maintains the package, for us it's usually the development team
(03:32:51 PM) Laney: XSBC-... - for packages created in Ubuntu first, the initial packager
(03:32:58 PM) Laney: for packages which come from Debian, the Debian maintainer
(03:33:09 PM) Laney: Build-Depends: packages which must be installed for this one to build
(03:33:20 PM) Laney: Standards-Version: version of debian policy which this package conforms to
(03:33:33 PM) Laney: Homepage: upstream homepage for software
(03:33:43 PM) AntoineLeclair: shadeslayer: QUESTION : How does one determine dependencies?
(03:33:48 PM) Laney: shadeslayer: We'll come to this
(03:34:08 PM) Laney: after the blank line, the next lines refer to the *binary* package
(03:34:29 PM) Laney: we are working with a source package currently; the binary package is what we build at the end
(03:34:32 PM) Laney: (.deb)
(03:34:46 PM) Laney: there can be multiple binary stanzas
(03:34:52 PM) Laney: Package: name of the binary package
03:35:13 PM) Laney: Architecture: CPU architectures for which this package works (can be - and usually is - 'all')
(03:35:35 PM) Laney: Depends: packages which must be installed for this one to work
(03:35:47 PM) Laney: Description: self explanatory - displayed in various pieces of software
(03:35:58 PM) Laney: there are other fields, but these are the ones we need here
(03:36:06 PM) Laney: See http://www.debian.org/doc/debian-policy/ch-controlfields.html for more
(03:36:30 PM) Laney: Usually I copy the control file from another package and edit it to suit
(03:36:43 PM) Laney: so now please open debian/control in your editor
(03:37:47 PM) Laney: to speed this up I've filled in most of the info
(03:37:52 PM) Laney: paste these contents in http://pastebin.com/m58074002
(03:38:30 PM) Laney: The most important thing to figure out are the build dependencies
(03:38:43 PM) Laney: the first thing we definitely need is debhelper
(03:38:50 PM) Laney: as this is what we are using to build the package
(03:39:02 PM) Laney: so please change line 6 to Build-Depends: debhelper (>= 7)
(03:39:12 PM) Laney: this says that we need debhelper of at least version 7 installed to build
(03:39:20 PM) AntoineLeclair: mruiz: QUESTION: Why Ubuntu Developers as Maintainers? What about MOTU Developers ?
(03:39:34 PM) Laney: mruiz: in anticipation of the archive reorganisation when MOTU will be going away
(03:40:06 PM) Laney: to figure out the rest of the build-deps, we would usually look in the README
(03:40:18 PM) Laney: or INSTALL file, but for this software they are not so useful
(03:40:27 PM) Laney: so we will open up configure.ac
(03:40:44 PM) AntoineLeclair: AlanBell: QUESION: is it important/desireable to use an @ubuntu.com email address?
(03:40:58 PM) Laney: AlanBell: For the maintainer or original maintainer?
(03:41:12 PM) Laney: configure.ac is used by the GNU autotools to build the configure script
(03:41:34 PM) Laney: part of running ./configure is checking that the system has the necessary prerequisites installed
(03:41:46 PM) Laney: it requres some skill to understand this file
(03:42:23 PM) Laney: but what we can understand from it is that we need gtkmm-2.4 greater than or equal to 2.4 installed and intltool greater than or equal to 0.35
(03:42:33 PM) Laney: this is one of the skills that you will develop when you maintain packages
(03:43:26 PM) Laney: so, please add ", libgtkmm-2.4 (>= 2.8), intltool (>= 0.35.0)" to your build-dep line
(03:43:39 PM) Laney: save and quit this file
(03:44:02 PM) Laney: er, wait, please don't do that ;)
(03:44:05 PM) Laney: reopen debian/control
(03:44:18 PM) Laney: we need to sort out the dependencies for the binary package
(03:44:21 PM) Laney: this is the Depends: line
(03:44:33 PM) Laney: this should be Depends: ${shlibs:Depends}, ${misc:Depends}, pdftk
(03:44:57 PM) Laney: what this says is to insert the dependencies for the shared libraries (resolved by dh_shlibdeps - see man page for more)
(03:45:17 PM) Laney: other parts of the build process can insert their own dependencies, which will be added to misc:Depends
(03:45:31 PM) Laney: these are called substvars, short for substitution variables
(03:45:56 PM) Laney: and pdftk is the application which pdfchain uses to transform pdfs, but won't be detected by shlibs because it is not a linked library
(03:46:02 PM) Laney: it is called as a system binary
(03:46:05 PM) Laney: *now* you can save and quit
(03:46:18 PM) Laney: invoke debuild -S -us -uc to build the source package
(03:46:39 PM) Laney: cd ..
(03:46:42 PM) Laney: ls *.dsc
(03:46:46 PM) Laney: you should see the source package we just made
(03:46:50 PM) Laney: which you can build with pbuilder
(03:46:55 PM) Laney: pbuilder-karmic build *.dsc
(03:48:22 PM) Laney: pbuilder-disc karmic build *.dsc if you did not build your pbuilder with symlinks
(03:49:27 PM) Laney: if you do not have pbuilder installed
(03:49:33 PM) Laney: cd pdfchain-0.123
(03:49:51 PM) Laney: sudo apt-get install libgtkmm-2.4-dev intltool
(03:50:03 PM) Laney: (and debhelper if you dont have this)
(03:50:08 PM) Laney: fakeroot debian/rules binary
(03:50:47 PM) Laney: We're rapidly running out of time, so I'm going to speed this up a lot
(03:50:55 PM) Laney: the build will fail in the documentation step
(03:51:02 PM) Laney: test step, sorry
(03:51:14 PM) Laney: this is because the upstream make check rule is broken
(03:51:26 PM) Laney: this should be reported as a bug to upstream - someone please feel free to do this
(03:52:16 PM) Laney: we are going to disable the test for now
(03:52:38 PM) Laney: Make your debian/rules look like: http://pastebin.com/f4d339347
(03:53:04 PM) Laney: this says to run the following commands instead of dh_auto_test, which is the debhelper command that runs the tests
(03:53:14 PM) Laney: the following commands are nothing ;)
(03:53:40 PM) Laney: overrides were a feature introduced in debhelper version 7.0.50, so we need to change ">= 7 to >= 7.0.50" in the build-deps line
(03:53:59 PM) Laney: now the package will build successfully :)
(03:54:34 PM) Laney: However, there is still an upstream bug where the documentation is placed in /usr/doc instead of /usr/share/doc
(03:54:39 PM) AntoineLeclair: randomaction: shouldn't we use libgtkmm-2.4-dev for build-depnds?
(03:54:43 PM) Laney: randomaction: yes
(03:54:52 PM) Laney: that's what I meant, sorry
(03:54:59 PM) Laney: See http://pastebin.com/f5a2622fe
(03:55:26 PM) Laney: dh_install is the debhelper command which deals with installing files into packages
(03:55:33 PM) Laney: we will patch this rule to move the documentation to the right place
(03:55:45 PM) Laney: http://pastebin.com/f547f4598
(03:56:17 PM) Laney: this says to move the files from debian/pdfchain/usr/doc to debian/pdfchain/usr/share/doc
(03:56:21 PM) Laney: which is the correct location for them
(03:56:33 PM) Laney: debian/pdfchain is where the files are built to before being placed into your .deb file
(03:56:57 PM) Laney: OK we're pretty much out of time so I'm going to give oyu links to read the stuff I didn't get to
(03:57:11 PM) Laney: sorry for having to go so fast, I hope you managed to get something
(03:57:19 PM) Laney: We still need to:
(03:57:45 PM) Laney:   - Fill out a package description. See http://www.debian.org/doc/debian-policy/ch-controlfields.html#s-f-Description for more on this
(03:58:37 PM) Laney:   - Fill out the debian/copyright file. See http://www.debian.org/doc/debian-policy/ch-docs.html#s-copyrightfile and http://dep.debian.net/deps/dep5/
(03:58:55 PM) Laney: For reference, http://pastebin.com/f75b86ac8 is the file I came up with
(03:59:21 PM) Laney: and you can get the final version of the pdfchain package from my PPA at https://edge.launchpad.net/~laney/+archive/ppa
(03:59:48 PM) Laney: Please ask any questions you have on this in #ubuntu-motu, and sorry again for having to rush
(03:59:58 PM) Laney: as you can see, there is a lot to know when packaging from scratch :)