Ubuntu Wiki

This page is based on the MOTU/School/PackagingWithoutCode session presented by Emmet Hikory on 2008-06-03.

Introduction

The purpose of this guide is to explain packaging of those special types of package that don't have any source. The important thing to consider is how the typical meanings of "source package" and "binary package" apply in these cases. There typically isn't anything to compile, and you just want to put the files on the target system. So, at a high level, a "source" package is the set of files that is used to edit and maintain a package. A "binary" package is the set of files that get installed on a user system. Even where most of the files are the same, it is useful to maintain this distinction, because there may be some files (README, COPYING, etc.) that don't need to be installed on a user system. Also, there may be special configuration changes that need to be applied to a user system, but which are not required for development. By keeping separate source and binary packages, it becomes very easy to separate these files into the appropriate places, and improves the experience for both users and developers. This is true even though the binary package may not have any actual binaries.

Preparing the Source

As an example, we're going to package the very simple script from

#! /bin/bash

echo Hello
$HOST=$(cat /etc/hostname)
echo Welcome to $HOST

Now, the first thing to notice is that upstream didn't license this script properly, so we need to ask upstream for a license. So upstream finally provides the following license, permitting us to package the script like this:

#! /bin/bash

# Copyright 2008 Emmet Hikory <persia@ubuntu.com>

# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.

# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

echo Hello
$HOST=$(cat /etc/hostname)
echo Welcome to $HOST

If you are the upstream, then you should give your own package a license. As upstream just sent us the script in a pastebin, and as personal scripts will normally just be sitting in text files, we'll need to create an orig.tar.gz ourselves.

Therefore, we create a directory hello-host-0.1. Notice that the directory is named "package-version", whereas the upcoming orig tarball will be named "package_version.orig.tar.gz." And we copy the script into the directory, and call it hello-host. We then tar and compress the directory, and get hello-host-0.1.tar.gz. This needs to be renamed to hello-host_0.1.orig.tar.gz. After we have the orig.tar.gz, we can start the packaging.

Populating debian/

We'll create a debian/ directory in the root of the directory that contains the script.

debian/copyright

Our first target will be copyright. This needs to include information about the package license. As this package doesn't have a homepage, we can get away with something simple.

This package was debianized by Emmet Hikory <persia@ubuntu.com> on Wed,  4 Jun 2008 00:44 +0900

Upstream Author: Emmet Hikory <persia@ubuntu.com>

Copyright: 2008 Emmet Hikory

License:
    Permission to use, copy, modify, and/or distribute this software for any
    purpose with or without fee is hereby granted, provided that the above
    copyright notice and this permission notice appear in all copies.

    THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
    WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
    MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
    ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
    WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
    ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
    OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

The Debian packaging is copyright 2008 Emmet Hikory <persia@ubuntu.com> and available on the same terms as the package itself (see above).

debian/control

Next, we'll need a debian/control. This is again fairly simple. Just use the minimum information. The following is a quick debian/control for our package:

Source: hello-host
Section: text
Priority: optional
Maintainer: Emmet Hikory <persia@ubuntu.com>
Build-Depends: cdbs
Standards-Version: 3.7.2

Package: hello-host
Architecture: all
Depends: ${misc:Depends}
Description: Short example program
 hello-host is an example script that displays the hostname

There are two stanzas, the source stanza and the binary stanza. The source stanza describes the source package, and the binary stanza the binary package. For the source stanza, you need to set the Source: header to be the name of the package. The Section header should match one of the package sections: http://www.debian.org/doc/debian-policy/ch-archive.html#s-subsections. Priority is almost always "optional" for new packages, although it can be "extra" if it breaks something (try not to do this). The Maintainer is the person responsible for the package. Standards-Version: should match whatever the current standards are when you package it.

Similarly, as we will describe below, this package will be built with CDBS, so it only Build-Depends on CDBS. Although CDBS uses debhelper but depends on it, you don't have to add debhelper in the Build-Depends. If you use a feature from a newer debhelper than the version depended upon by CDBS, you can add a versioned depends. It's a good idea to add a debian/compat file containing any of "5", "6", or "7" depending on the features of debhelper that you need. In which case, you want to add "debhelper (>= 5)" (or 6 or 7) to your Build-Depends.

Now, for the binary stanza, we again have the package name. Scripts are always Architecture: all. As mentioned previously, you need to manually set the Depends: depending on what the scripts require. I tend to always include ${misc:Depends} just in case some of the CDBS magic requires something.

Note that because it's just scripts, and we're not using any special handlers, we have to set all the Dependencies manually. In this case, it's a bash script, so we should depend on bash. However, since bash is marked as Priority: Required, we do not need to make this script depend on it. For something like a PHP script, it is much more important to set the dependencies correctly. Sometimes, it will be necessary to specificy that a script only works with a newer version of a package, so you should set the depends to: package (>= version).

More information about writing control files is available from http://www.debian.org/doc/debian-policy/ch-controlfields.html

debian/rules

Now we move on to debian/rules. I recommend using CDBS for this type of package, only because it means less typing, although those familiar with debhelper 7 may find that as easy a choice. I use this mostly because I don't want to think about it, and will use the implied debhelper calls for most things. So, for this example, I'm using CDBS, so debian/rules looks like this:

#! /usr/bin/make -f

include /usr/share/cdbs/1/rules/debhelper.mk

debian/install

Now comes the important part: this needs to be installed somewhere. This is handled by the implied call to dh_install. We create a debian/install file that specifies where the files go. Here is an example:

hello-host usr/bin

• Note that dh_install will automatically create the usr/bin directory in the package tree to contain these files once the location is properly specified.

debian/changelog

Finally, create a changelog with dch -create (this is in the devscripts package) that briefly describes the initial packaging of the script, and if necessary, the bugs it closes.

Next Steps

Now we have a package that installs the scripts into the right places, and has all the necessary bits to make a good package. There are still a couple pieces missing: we probably want to create a manual page, as otherwise lintian will complain, and we'll never get it past REVU.

If we're packaging a firmware installer, or something that needs to initialise a database, we might want to do something with debconf or maintainer scripts. If you get as far as trying to package an entire Ruby-on-Rails application, don't forget to install the necessary integration hooks for the webservers, and handle the DB startup and cleanup in the maintainer scripts.

Sources for Scripts to Package

If you are looking for scripts to package, you can try out some of the following:

http://bashscripts.org/ has heaps of samples, although many are not suitable for packaging

http://www.scripts.com/ has some python and perl (but be careful, some of these may be more than scripts)

On the other hand, aside from experimentation (for which variations on hello-host work just fine), you'd do best to wait until you found something so useful you didn't know how you worked without it before packaging it for general release.

Handling upstream version changes

This method was adapted from http://www.eyrie.org/~eagle/notes/debian/build-tools.html#scripts.

Unfortunately, using a debian/watch file with a script to watch for upstream changes is not likely to work. Since many scripts will just be stored on a server in plaintext form, rather than as tarballs, you can't search through hrefs using debian/watch to find and download a new .orig tarball. You can solve this by adding an extra target to debian/rules:

PACKAGE = upstream-name
URL     = http://upstream/upstream/location
VERSION = `awk '/Id: / { print $$6; exit }' $(PACKAGE)`

get-orig-source:
 wget -O $(PACKAGE) $(URL)
 chmod 755 $(PACKAGE)
 mkdir $(PACKAGE)-$(VERSION)
 cp -p $(PACKAGE) $(PACKAGE)-$(VERSION)/
 tar cfz $(PACKAGE)_$(VERSION).orig.tar.gz $(PACKAGE)-$(VERSION)
 rm -r $(PACKAGE)-$(VERSION)

The above version line uses a CVS Id for versioning. If another method is used, then change the variable accordingly.

Using this target, you can call "fakeroot debian/rules get-orig-source." This will fetch the new source of the script from the server and rebuild the .orig.tar.gz with the new version of the script.

TODO

(Note that those preferring machine-readable debian/copyright are welcome to do it that way) (in fact, if someone wants to draft that, and post for the rest of the class, it would be appreciated)

Descriptions about packaging specific kinds of scripts might be helpful as well.

Original IRC Chatlog

[16:30] <persia> Welcome to the special session on packaging scripts, webapps, firmware installers, and other things without code.
[16:33] <persia> So, this is about packaging those special types of package that don't have any source.
[16:33] <persia> The important thing to consider is how the typical meanings of "source package" and "binary package" apply in these cases.
[16:34] <persia> There typically isn't anything to compile, and you just want to put the files on the target system.
[16:34] <persia> So, at a high level, a "source" package is the set of files that is used to edit and maintain a package.
[16:34] <persia> A "binary" package is the set of files that get installed on a user system.
[16:35] <persia> Even where most of the files are the same, it is useful to maintain this distinction, because there may be some files (README, COPYING, etc.) that don't need to be installed on a user system.
[16:35] <persia> Also, there may be special configuration changes that need to be applied to a user system, but which are not required for development.
[16:36] <persia> By keeping separate source and binary packages, it becomes very easy to separate these files into the appropriate places, and improves the experience for both users and developers.
[16:36] <persia> This is true even though the binary package may not have any actual binaries.
[16:38] <persia> As an example, we're going to package the very simple script from http://paste.ubuntu.com/16605/
[16:38] <persia> Now, the first thing to notice is that upstream didn't license this script properly, so we need to ask upstream for a license.
[16:40] <persia> So upstream finally provides http://paste.ubuntu.com/16607/
[16:40] <persia> This, we can package.
[16:40] <persia> as upstream just sent us the script in a pastebin, we'll need to create an orig.tar.gz ourselves.
[16:41] <persia> So we create a directory hello-host-0.1
[16:41] <persia> And we copy the script into the directory, and call it hello-host.
[16:41] <persia> We then tar and compress the directory, and get hello-host-0.1.tar.gz
[16:42] <persia> This needs to be renamed to hello-host_0.1.orig.tar.gz.
[16:42] <persia> After we have the orig.tar.gz, we can start the packaging.
[16:42] <persia> First, we'll create a debian/ directory.
[16:42] <persia> Our first target will be copyright.
[16:43] <persia> This needs to include information about the package license.
[16:43] <persia> As this package doesn't have a homepage, we can get away with something simple.
[16:46] <persia> http://paste.ubuntu.com/16608/
[16:46] <persia> (Note that those preferring machine-readable debian/copyright are welcome to do it that way)
[16:46] <persia> (in fact, if someone wants to draft that, and post for the rest of the class, it would be appreciated)
[16:47] <persia> Next, we'll need a debian/control.  This is again fairly simple.  Just use the minimum information.
[16:47] <persia> I recommend using CDBS for this type of package, only because it means less typing, although those familiar with debhelper 7 may find that as easy a choice.
[16:49] <persia> http://paste.ubuntu.com/16609/ is a quick debian/control for our package.
[16:49] <persia> Note that because it's just scripts, and we're not using any special handlers, we have to set all the Dependencies manually.
[16:49] <persia> In this case, it's a bash script, so we depend on bash.
[16:50] <sebner> persia: is that necessary?
[16:51] <persia> sebner: I'm actually not sure.  I know bash is no longer the default /bin/sh, but I don't know if it is still Priority: Essential.
[16:51] <persia> For something like a PHP script, it is much more important.
[16:52] <persia> So, for this example, I'm using CDBS, so debian/rules looks like http://paste.ubuntu.com/16610/
[16:53] <persia> I use this mostly because I don't want to think about it, and will use the implied debhelper calls for most things.
[16:53] <persia> Now comes the important part: this needs to be installed somewhere.
[16:53] <persia> This is handled by the implied call to dh_install.
[16:54] <persia> We create a debian/install file that specifies where the files go.  Our example is http://paste.ubuntu.com/16611/
[16:54] <persia> Now this directory doesn't exist yet, so we need a debian/dirs to create it http://paste.ubuntu.com/16613/
[16:55] <persia> It is important not to have the leading / to make sure that it installs in the package, instead of the developer's system.
[16:56] <persia> Now we have a package that installs the scripts into the right places, and has all the necessary bits to make a good package.
[16:57] <persia> There are still a couple pieces missing: we probably want to create a manual page, as otherwise lintian will complain, and we'll never get it past REVU.
[16:58] <persia> If we're packaging a firmware installer, or something that needs to initialise a database, we might want to do something with debconf or maintainer scripts.
[17:02] <persia> If you get as far as trying to package an entire Ruby-on-Rails application, don't forget to install the necessary integration hooks for the webservers, and handle the DB startup and cleanup in the maintainer scripts.
[17:02] <persia> sebner: Yep.  Packaging scripts is pretty easy.  Once the files are installed in the right place, the user is typically happy.
[17:03] <schmiedc> so but how do i know what i have to write in the certain scripts
[17:03] <persia> The really tricky bit is understanding the difference between a "source" package and a "binary" package when the "binary" package looks like it contains source.
[17:03] <persia> schmiedc: Which scripts?
[17:04] <schmiedc> for example the debian/control
[17:04] <persia> schmiedc: OK.  We created debian/copyright, debian/control, debian/rules, debian/changelog, and debian/install.
[17:05] <persia> debian/rules can almost always be just the two lines for script packages.
[17:05] <persia> debian/install is set up based on where the files need to end up.  This is usually available in the upstream documentation.
[17:05] <persia> Oh, and I forgot about debian/dirs.
[17:06] <persia> debian/dirs needs to include all the directories used in debian/install.
[17:06] <persia> debian/changelog is best created with dch --create
[17:07] <persia> debian/copyright is often just a quick summary of: who packaged it and when, the authors, the copyright holders, the upstream license, and the packaging license.
[17:07] <persia> debian/control is a bit trickier.  Let's look at http://paste.ubuntu.com/16609/ carefully.
[17:07] <persia> There are two stanzas, the source stanza and the binary stanza
[17:08] <persia> The source stanza describes the source package, and the binary stanza the binary package.
[17:08] <persia> For the source stanza, you need to set the Source: header to be the name of the package.
[17:08] <persia> The Section header should match one of the package sections
[17:09] <persia> http://www.debian.org/doc/debian-policy/ch-archive.html#s-subsections
[17:09] <persia> Priority is almost always "optional" for new packages, although it can be "extra" if it breaks something (try not to do this)
[17:09] <persia> The Maintainer is the person responsible for the package.
[17:10] <persia> Build-Depends should be "cdbs": you shouldn't need anything else for a simple scripts package.
[17:10] <albert23> should debhelper be added?
[17:10] <persia> Standards-Version: should match whatever the current standards are when you package it.
[17:11] <persia> albert23: It's not required, as cdbs depends on debhelper.
[17:11] <persia> If you use a feature from a newer debhelper than the version depended upon by CDBS, you can add a versioned depends.
[17:12] <persia> albert23: You remind me, it's a good idea to add a debian/compat file containing any of "5", "6", or "7".
[17:12] <persia> In which case, you want to add "debhelper (>= 5)" (or 6 or 7) to your Build-Depends.
[17:13] <persia> Now, for the binary stanza, we again have the package name
[17:13] <persia> scripts are always Architecture: all
[17:14] <persia> As mentioned previously, you need to manually set the Depends: depending on what the scripts require.  I tend to always include ${misc:Depends} just in case some of the CDBS magic requires something.
[17:14] <persia> And the Description is as normal.
[17:14] <persia> more information about writing control files is available from http://www.debian.org/doc/debian-policy/ch-controlfields.html
[17:15] <schmiedc> but architecture would be i368 and so on or?
[17:15] <persia> schmiedc: As these are just scripts, it ought be architecture: all.  That way they can work for anyone, regardless of their architecture (as long as they meet the Depends)
[17:16] <persia> schmiedc: Oh, sure, for non-script packages, you might need to set architectures.  "any" is a good choice.
[17:18] <mok0> I think, for consistency, it is better to name install and dirs -> <package>.install and <package>.dirs
[17:19] <persia> mok0: It works either way, but sure.
[17:19] <schmiedc> or to mention in the beginning to replace debian/ with the packages name
[17:20] <persia> schmiedc: No, you use debian/$(pacakgename).install and debian/$(packagename).dirs, etc.
[17:22] <schmiedc> where is a good place to look for simple upstreams to play with?
[17:23] <persia> schmiedc: http://bashscripts.org/ has heaps of samples, although many are not suitable for packaging
[17:24] <persia> http://www.scripts.com/ has some python and perl (but be careful, some of these may be more than scripts)
[17:25] <persia> On the other hand, aside from experimentation (for which variations on hello-host work just fine), you'd do best to wait until you found something so useful you didn't know how you worked without it before packaging it for general release.
[17:29] <persia> Thanks to everyone for attending, even with the short notice.  Good luck with your script packaging, and don't hesitate to ask in #ubuntu-motu if you have any further questions.

These logs have been slightly edited for clarity.