Ubuntu Wiki

Open Week -- Ubuntu Development II -- Daniel Holbach -- Fri, May 7

EDT -5

(12:01:48 PM) dholbach: ok... I'm back, tea almost ready :)
(12:02:04 PM) dholbach: how was part 1? everything OK so far? anything unclear?
(12:02:17 PM) dholbach: p3rg2z asks "is this session logged somewhere where I can access it later?"
(12:02:22 PM) enli|afk is now known as enli
(12:02:30 PM) dholbach: yes, it will be linked from https://wiki.ubuntu.com/UbuntuOpenWeek
(12:02:41 PM) dholbach: and also on http://irclogs.ubuntu.com
(12:03:06 PM) dholbach: any more questions?
(12:03:54 PM) dholbach: if not, let's crack on and let's see how ugly, unfair and bad packaging really is
(12:04:03 PM) ClassBot: Miroslav_RS asked: What is jcastro's cat called?
(12:04:19 PM) dholbach: Miroslav_RS: good one - I believe it's called Miguel :)
(12:04:48 PM) dholbach: to get us in the packaging business, please run
(12:04:52 PM) dholbach:   apt-get source hello-debhelper
(12:05:15 PM) dholbach: (this should work if you checked this tickbox: System → Administration → Software Sources → Ubuntu Software → Enable "Sources"
(12:05:18 PM) dholbach: )
(12:05:40 PM) ClassBot: plod63 asked: if we have questions after we try this ourselves (sorry i know i should be doing it now but my vm isnt installing fast enough, what is the best place to ask them)?
(12:05:49 PM) dholbach: plod63: #ubuntu-packaging and #ubuntu-motu
(12:05:55 PM) dholbach: or ubuntu-motu-mentors@lists.ubuntu.com
(12:06:31 PM) dholbach: ok... I have three new files lying around now:
(12:06:31 PM) dholbach: ello-debhelper_2.4-3.diff.gz  hello-debhelper_2.4-3.dsc  hello-debhelper_2.4.orig.tar.gz
(12:06:37 PM) dholbach: sorry, make that:
(12:06:40 PM) dholbach: hello-debhelper_2.4-3.diff.gz  hello-debhelper_2.4-3.dsc  hello-debhelper_2.4.orig.tar.gz
(12:07:08 PM) dholbach: .orig.tar.gz is the file that somebody got directly from the homepage of the developers of hello
(12:07:41 PM) dholbach: the .diff.gz contains changes we make to make the package build the debian/ubuntu way (the packaging)
(12:07:57 PM) dholbach: and the .dsc file is a piece of text with meta data like checksums and the like
(12:08:13 PM) dholbach: the three files together are what we call the "source package"
(12:08:33 PM) dholbach: that's what you download to modify it and look at the source or upload to get it built in launchpad
(12:09:27 PM) dholbach: recently some changes in the debian/ubuntu packaging world allow different kinds of source packages, but in essence it's "the same": unmodified upstream tarball, bunch of changes, meta-data
(12:10:05 PM) dholbach: "apt-get source" (actually dpkg-source) was kind enough to unpack the tarball and apply the changes for us
(12:10:11 PM) dholbach: so let's have a look what we have here
(12:10:20 PM) dholbach:   cd hello-debhelper-2.4
(12:10:48 PM) dholbach: if you have a look what's in the directory, you find the usually stuff you have when you compile software
(12:11:00 PM) dholbach: a README file, a configure script and the like
(12:11:12 PM) dholbach: what's new is the debian/ directory, which is where the packaging goes
(12:11:25 PM) dholbach: it contains these files here:
(12:11:26 PM) dholbach:   changelog  compat  control  copyright  rules
(12:11:36 PM) dholbach: if you
(12:11:40 PM) dholbach:   less debian/changelog
(12:12:08 PM) dholbach: it will show you the history of the last hello-debhelper releases
(12:12:26 PM) dholbach: the entries always stick to the same format:
(12:12:58 PM) dholbach: <source package name> (<version>) <debian/ubuntu release>; urgency=<some urgency>
(12:13:12 PM) dholbach: then some bullet points about what changed in that revision
(12:13:32 PM) dholbach: then the name and email address who made the changes and the timestamp
(12:13:44 PM) dholbach: <source package name> in our case is hello-debhelper
(12:13:58 PM) dholbach: <version> in my case is 2.4-3
(12:14:06 PM) dholbach: let's have a look at the version more closely
(12:14:25 PM) dholbach: "2.4" means that 2.4 was released by the upstream developers, the software authors
(12:14:46 PM) dholbach: and the "-3" means that three revisions of this version were uploaded to Debian
(12:15:22 PM) dholbach: if now upstream decided to release 2.5 and I was to package it, I'd upload 2.5-0ubuntu1 to Ubuntu
(12:15:50 PM) dholbach: which would mean 2.5 released upstream, 0 revisions of it in Debian, first revision in Ubuntu
(12:16:00 PM) dholbach: (I hope that was clear enough :-))
(12:16:48 PM) dholbach: <debian/ubuntu release> was "unstable" because that's where the debian maintainer uploaded it
(12:16:57 PM) dholbach: I'd upload to "maverick" instead
(12:17:06 PM) dholbach: in Ubuntu you can only upload to the current development release
(12:18:00 PM) dholbach: there are ways to get packages backported or bugs fixed in <old-release>-updates, but that's side-cases :)
(12:18:38 PM) dholbach: alright, I think that should be enough for debian/changelog :)
(12:19:43 PM) dholbach: debian/compat is quite boring, as it merely controls the behaviour of the debhelper scripts used (more on that later), but it's safe to ignore that for now
(12:20:04 PM) dholbach: debian/control is one of the heart pieces of the package, so let's have a look at it now
(12:20:28 PM) ClassBot: akk asked: in "2.5-0ubuntu1" why the 1 after ubuntu?
(12:21:23 PM) dholbach: akk: 2.5-0ubuntu1 means: the software authors released version 2.5, "-0" means that it wasn't introduced in Debian yet, "ubuntu1" means it's the first revision of it in Ubuntu
(12:21:41 PM) dholbach: so 2.5-0ubuntu2 would be the same, but the second revision of it in Ubuntu
(12:22:11 PM) dholbach: basically it's     X-YubuntuZ    for packages we decide to modify in Ubuntu (as opposed to packages we inherit as they are from Debian)
(12:22:42 PM) dholbach: (and ignoring the case of native packages, which we'll ... ignore for now :-))
(12:22:58 PM) dholbach: back to debian/control:
(12:23:06 PM) dholbach: it always consists of at least 2 stanzas
(12:23:16 PM) dholbach: first one describes the source package
(12:23:29 PM) dholbach: all the following ones describe the resulting binary (.deb) packages
(12:25:09 PM) dholbach: source packages can build more than one binary package
(12:25:35 PM) dholbach: for example you could have a nice tool that has a massive amount of documentation
(12:25:44 PM) dholbach: in that case you'd probably want to create
(12:25:53 PM) dholbach:  nice-tool  and  nice-tool-doc
(12:26:04 PM) dholbach: so you don't overburden people who JUST WANT THE tool :)
(12:26:45 PM) dholbach: over here the source entry looks like this:
(12:26:46 PM) dholbach: Source: hello-debhelper
(12:26:46 PM) dholbach: Section: devel
(12:26:46 PM) dholbach: Priority: extra
(12:26:46 PM) dholbach: Maintainer: Santiago Vila <sanvila@debian.org>
(12:26:46 PM) dholbach: Standards-Version: 3.8.3
(12:26:47 PM) dholbach: Build-Depends: debhelper (>= 7)
(12:28:06 PM) dholbach: you define the name of the project, place it in some section (the debian policy knows all the valid sections), give it a Priority (defined in the debian policy too), you tell it who the maintainer is (as I said before that's less important in the ubuntu world), you specify the Standards-Version (version of the debian policy the package complies with) ...
(12:28:13 PM) dholbach: and you mention the Build-Depends
(12:28:27 PM) dholbach: this is the list of packages that are required to compile the package
(12:28:33 PM) dholbach: in our case it's a very small list, it's just debhelper
(12:28:48 PM) dholbach: (build-essential is always required to be there, so it doesn't need to be mentioned explicitly)
(12:29:14 PM) dholbach: in the source section, you will, apart from the Build-Depends, rarely make changes
(12:29:47 PM) dholbach: so let's proceed to our binary package stanza
(12:29:52 PM) dholbach: the first part of it is:
(12:29:52 PM) dholbach: in the source section, you will, apart from the Build-Depends, rarely make changes
(12:29:55 PM) dholbach: oops
(12:29:58 PM) dholbach: wrong paste :)
(12:30:04 PM) dholbach: Package: hello-debhelper
(12:30:04 PM) dholbach: Architecture: any
(12:30:04 PM) dholbach: Depends: ${shlibs:Depends}, ${misc:Depends}
(12:30:11 PM) dholbach: the package name is defined too
(12:31:04 PM) dholbach: now we specify for which architectures the package is built
(12:31:26 PM) dholbach: there are two very common values for "Architecture" that regularly cause confusion :)
(12:31:31 PM) dholbach: in our case it's "any"
(12:31:53 PM) dholbach: which means: build this package for each and every architecture individually
(12:32:24 PM) dholbach: so we'll have a different package for every architecture later on: i386, amd64, powerpc, sparc, arm, lpia, ia64, etc
(12:32:40 PM) dholbach: the other very common value is "all"
(12:33:06 PM) dholbach: which means: this package has no architecture-dependent bits, build it once, use same package on all architectures
(12:33:30 PM) dholbach: for example if you have a couple of shell scripts in the package, or a few pictures
(12:33:44 PM) dholbach: they are always going to be the same, no matter on which architecture you use the package
(12:34:13 PM) dholbach: but you could also specify specific architecture where the package will be built
(12:35:09 PM) dholbach: Conflicts/Replaces/Provides are set to "hello" which means that dpkg won't allow you to install the hello and hello-debhelper package at the same time (they both contain usr/bin/hello and would overwrite each other)
(12:35:54 PM) dholbach: there's a lot you can do wrong with conflicts and replaces, that's why that part in the debian policy is a very good read :)
(12:36:16 PM) dholbach: following that we have a short description (first line) and a long description following that
(12:36:35 PM) dholbach: this is what "apt-cache show ...", synaptic and all the others will show you
(12:36:43 PM) dholbach: and that for a few releases now is translatable
(12:37:18 PM) dholbach: there are heaps of other settings you could make in debian/control, but I think what we see here is a good start
(12:37:27 PM) dholbach: do we have any questions about these=?
(12:38:17 PM) dholbach: ok, then please let me get back to versions once again, this short discussion happened in #ubuntu-classroom-chat a few moments ago
(12:38:28 PM) dholbach: <plod63> how do you know if its been released in debian yet or not, should you be monitoring there first?
(12:38:29 PM) dholbach: <akk> I think I was partly confused by the debian numbers starting from 0 but the ubuntu numbers starting from 1.
(12:39:12 PM) dholbach: akk and plod63 ask great questions - basically it boils down to: how do Debian and Ubuntu collaborate? how is code shoved around?
(12:39:56 PM) dholbach: if you have a quick look at http://wiki.ubuntu.com/ReleaseSchedule you will see how the Ubuntu releases are structured
(12:40:42 PM) dholbach: basically every 6 months we start working on a new release, the most important packages are uploaded first that form the build-chain (compilers, linker, etc.)
(12:41:10 PM) dholbach: then "we open the gates" and "sync" source packages from Debian
(12:41:47 PM) dholbach: this means that for every unmodified Ubuntu package we import the new Debian package and automatically build it
(12:41:52 PM) dholbach: an example:
(12:42:06 PM) dholbach: let's say we have gedit 2.3-1 in Ubuntu in the old release
(12:42:18 PM) dholbach: (2.3 released upstream, first revision in debian, no ubuntu changes)
(12:42:34 PM) dholbach: once we opened the new release, there' 2.4-3 available in Debian now
(12:42:44 PM) dholbach: (2.4 released upstream, 3 revisions in Debian)
(12:42:58 PM) dholbach: then the source of 2.4-3 will be imported and automatically built for us
(12:43:30 PM) dholbach: this happens until DebianImportFreeze (look at the schedule), from there on "syncs" need to be excplicitly requested
(12:43:42 PM) dholbach: here's another example:
(12:44:14 PM) dholbach: we have gnome-terminal 2.5-6ubuntu2 in Ubuntu and in Debian we have 2.7-3 now
(12:44:37 PM) dholbach: in that case, because we have Ubuntu modifications, the source will not be automatically overwritten
(12:45:28 PM) dholbach: we either need to merge changes (make sure our and Ubuntu and Debian changes are still in the new version - this would be 2.7-3ubuntu1 ok?) or decide we drop our changes and import 2.7-3 from Debian
(12:45:42 PM) ClassBot: plod63 asked: so this building happens until the freeze automatically?
(12:47:07 PM) dholbach: plod63: building almost always happens automatically (not when we're in pre-release freezes), here it's the import (of unmodified packages) that is automatic
(12:47:53 PM) dholbach: so you can imagine that the first part of the release cycle is all about merging/syncing changes in
(12:48:17 PM) dholbach: and that's usually a good opportunity to go through the list of our patches and decide if they're still worth it and make sure that they get forwarded to either Debian or upstream
(12:49:00 PM) dholbach: sometimes we decide to carry our own patches, even if Debian or upstream don't like them, but we want good reasons for that
(12:49:10 PM) dholbach: in all the other cases we forward those patches
(12:49:26 PM) dholbach: I hope with that information the pieces now fit a bit together :)
(12:49:53 PM) dholbach: ok, let's crack on: while we have a look at the rest of the package, please open another terminal and run the following
(12:50:09 PM) dholbach:   sudo pbuilder build hello-debhelper_*.dsc
(12:50:17 PM) dholbach: so we get at least one build done today :)
(12:50:38 PM) ClassBot: deuxpi asked: how does or coordinates the merge/sync process?
(12:51:14 PM) dholbach: deuxpi: all developers - usually the most important packages are merged / synced / uploaded first
(12:51:23 PM) dholbach: http://merges.ubuntu.com lists packages that need a merge
(12:51:59 PM) dholbach: usually the last uploader is expected to do the merge again, but if you have a quick word with them and say that you are going to do it, that's totally fine
(12:52:05 PM) dholbach: just important to not duplicate work
(12:52:20 PM) dholbach: now let's check out debian/copyright
(12:52:40 PM) dholbach: debian/copyright usually lists:
(12:52:47 PM) dholbach:  - where the source was obtained form
(12:52:55 PM) dholbach:  - who has a copyright on the code
(12:53:01 PM) dholbach:  - which license the code is under
(12:53:04 PM) dholbach:  - who packaged it
(12:53:12 PM) dholbach: it's incredibly important to get this right
(12:53:55 PM) dholbach: you don't want code with an un-free license in your package or code that you are not allowed to modify, etc.
(12:54:14 PM) dholbach: if packages like that get into the archive, Ubuntu as a project would get into a lot of trouble
(12:54:37 PM) dholbach: luckily there's YOU who can make sure that all pieces of code are free and under a good license :)
(12:54:44 PM) dholbach: and luckily there's archive admins who double-check
(12:55:04 PM) dholbach: lastly, there's debian/rules
(12:55:36 PM) dholbach: beside having a quite funny name this file is one of the heart pieces of the source package also
(12:55:42 PM) dholbach: because it determines how the package it built
(12:56:06 PM) dholbach: those of your who have looked at Makefiles before will notice that it's a Makefile too
(12:56:22 PM) dholbach: it's split up into sections such as clean, build, install, etc.
(12:56:48 PM) dholbach: so what you do when you run "./configure && make" or "make clean", "sudo make install", etc. is run here too
(12:57:12 PM) dholbach: but run in a sane way, so you don't have millions of files written on your system :)
(12:57:44 PM) dholbach: you can also see all the dh_* calls in there, that's debhelper I was referring to the whole time - extremely useful packaging tools
(12:58:23 PM) dholbach: I couldn't have done packaging as a whole complete justice in these two sessions and you now only have a very quick overview of debian/rules
(12:58:41 PM) dholbach: I encourage you to read the source of other source packages and check out https://wiki.ubuntu.com/PackagingGuide too
(12:58:59 PM) dholbach: your package should have built by now and lie in /var/cache/pbuilder/result/ :)
(12:59:13 PM) dholbach: I hope to see you in #ubuntu-packaging and #ubuntu-motu soon
(12:59:22 PM) dholbach: go and bookmark https://wiki.ubuntu.com/MOTU/GettingStarted
(12:59:47 PM) dholbach: ROCK ON EVERYBODY and thanks for listening!