Ubuntu Wiki

Dev Week -- MOTU School - Working With Debian -- James Westby -- Mon, Feb 18

(02:01:35 PM) dholbach: next up are our Packaging Kings: James Westby & Nicolas Valcárcel
(02:01:47 PM) dholbach: who will talk to you about Ubuntu and Debian collaboration
(02:01:50 PM) james_w: \o/
(02:01:51 PM) dholbach: thanks a lot everybody
(02:01:57 PM) ***dholbach hugs james_w and nxvl_work
(02:01:57 PM) james_w: thanks Daniel.
(02:02:31 PM) nxvl_work: heh, 10 lines of hugs and love, that's why i love ubuntu
(02:02:31 PM) dholbach: the stage is yours :)
(02:02:37 PM) james_w: Firstly I'd like to apologise to Nicolas, as my oversight meant that he wasn't listed as co-presenter of this talk. Sorry Nicolas.
(02:03:02 PM) nxvl_work: james_w: don't worry, i add myself :D now i'm in there
(02:03:25 PM) james_w: We are going to talk today a little about collaboration between Debian and Ubuntu, and how you can make the process smoother.
(02:03:52 PM) dholbach left the room (quit: "Ex-Chat").
(02:04:00 PM) james_w: Firstly I want to say a little about what the relationship between Debian and Ubuntu is.
(02:04:11 PM) james_w: Debian is the base upon which Ubuntu is built.
(02:04:24 PM) james_w: Almost all packages in Ubuntu come from Debian, and most are used unchanged.
(02:04:37 PM) james_w: This means that Ubuntu owes a lot to Debian and should endeavour to maintain a good relationship with them.
(02:05:04 PM) james_w: There are however several differences in the way that Debian and Ubuntu are organised, and in the decisions that they have made on some issues that you need to understand in order to work with them most effectively.
(02:05:32 PM) james_w: The biggest difference is that in Debian all packages have a maintainer, which may be a team, who controls the package.
(02:05:57 PM) james_w: (Please feel free to ask any questions that you have as we go).
(02:06:19 PM) james_w: They generally make all the decisions about the package, and are normally the only ones to perform uploads of that package.
(02:06:23 PM) snhome: here or in #-chat?
(02:06:30 PM) nxvl_work: snhome: -chat
(02:06:51 PM) james_w: There are some moves away from this, but it remains the status quo.
(02:07:08 PM) james_w: This is different to Ubuntu where in general a package doesn't have a maintainer as such, it is just looked after by all contributors.
(02:07:30 PM) james_w: This difference has effects on both Ubuntu and Debian.
(02:07:52 PM) james_w: For Ubuntu it means that generally a contributor isn't intimately familiar with a package, and doesn't know the Debian maintainer's opinion on things.
(02:08:12 PM) james_w: For Debian maintainers it means that they generally don't know who to contact if they wish to discuss a package in Ubuntu.
(02:08:43 PM) james_w: The differences in some policy decisions can also make it difficult for a Ubuntu contributor to know whether what they are doing also applies to Debian as well.
(02:08:56 PM) james_w: This can be especially true for bugs, where differing versions of packages, or different dependencies in the chain can hide or expose bugs.
(02:09:04 PM) edosar: but how many of debian mainteiners are ubuntu packages ?
(02:09:27 PM) nxvl_work: edosar: ask you question on #ubuntu-classroom-chat please
(02:09:59 PM) fredre left the room (quit: "Leaving").
(02:10:25 PM) smarter_ is now known as smarter
(02:10:36 PM) james_w: edosar: I don't understand your question I'm afraid, if you could re-ask it in #ubuntu-classroom-chat that would be great.
(02:11:07 PM) james_w: It can be helpful to not consider Debian as a "whole," but more as a group of people who each work in their own little area.
(02:11:22 PM) james_w: You will find that if you contact most of them with a bug report, a patch, or a question they will be very friendly and helpful.
(02:11:29 PM) RainC1 is now known as RainCT
(02:11:44 PM) james_w: There are a few people that this wouldn't really apply to, but they are definitely in the minority.
(02:12:09 PM) james_w: You will probably find if you contact those ones in a friendly manner about a specific issue then they will probably still help you, but it can be a scary thing to do.
(02:12:20 PM) james_w: What I am trying to tell you here is not to let the actions of a noisy few spoil your opinions of the whole community.
(02:12:58 PM) james_w: In order to understand the best ways to share work between Ubuntu and Debian we need to understand how packages flow between them.
(02:13:14 PM) james_w: Ubuntu takes all source packages from Debian that aren't on a blacklist and automatically "syncs" them in to Ubuntu.
(02:13:26 PM) james_w: This means that the source package is taken unmodified and just rebuilt in the latest development version of Ubuntu.
(02:13:45 PM) james_w: There are certain things that may be done while building which mean that you may get different binary packages, but they do not alter the functionality, for instance https://wiki.ubuntu.com/DebianMaintainerField.
(02:13:56 PM) james_w: This process stops at "DebianImportFreeze".
(02:14:05 PM) james_w: In order to sync a packageafter that stage you must file a sync request, following the process at https://wiki.ubuntu.com/SyncRequestProcess.
(02:14:49 PM) james_w: Where Ubuntu has to make changes to the package a new upload is done.
(02:15:05 PM) james_w: This upload has a version number containing "ubuntu", which ensures that the package won't be synced without manual action, so that the changes are not lost.
(02:15:19 PM) james_w: When this has been done in the past and a new upload is made to Debian, the package will be eligible for "merging".
(02:15:40 PM) james_w: This means taking the two versions of the package and reconciling the changes so that you get the new changes from Debian, but also keep the Ubuntu changes. See https://wiki.ubuntu.com/UbuntuDevelopment/Merging for more information.
(02:15:52 PM) james_w: Sometimes when you go to merge a package you will find that the new Debian version includes all the changes that were made in Ubuntu.
(02:16:04 PM) james_w: In that case you perform a sync, meaning that again Ubuntu is using Debian's package for that software.
(02:16:24 PM) james_w: And we all cheer, as that is Free Software in action (and we have less work to do).
(02:16:49 PM) james_w: So, what are the ways in which we can share work with Debian?
(02:17:27 PM) james_w: There are a whole bunch of ways, for instance you can share patches, work together to package new upstream versions, or coordinate transitions or security fixes.
(02:17:38 PM) james_w: You can even become maintainer or co-maintainer of a package in Debian so that the package can always be used unmodified in both with less work.
(02:18:02 PM) james_w: When you are about to make some changes in Ubuntu you can look to see whether these apply to Debian as well, and if they do contact the Debian maintainers and propose that you work together on the issue, or just send them the result of your work.
(02:19:00 PM) james_w: It's always a good practice to coordinate your changes with the Debian maintainer, so you can check if they differ from her plans, or she is already working on the problem.
(02:19:26 PM) james_w: We want to avoid cases where incompatible changes are made in the two distributions, as that creates a massive headache for the future.
(02:20:04 PM) james_w: Now, we'll show you a couple of places you can use to find information about packages in Debian.
(02:20:19 PM) james_w: There are several places from which you can find information about packages in Debian. Knowing where to look to find something out can be quite an art in itself.
(02:20:32 PM) james_w: Luckily there is one place that tries to gather as much of this information as possible. That place is the Package Tracking System, known as the PTS.
(02:21:03 PM) james_w: The PTS lives at http://packages.qa.debian.org/. You can quickly go to the page for a package using http://packages.qa.debian.org/package.
(02:21:14 PM) james_w: Note that it works using the source package name, but if you enter a binary package name you get a helpful redirect page.
(02:21:29 PM) james_w: Let's take a look at a typical page: http://packages.qa.debian.org/gnupg.
(02:21:43 PM) james_w: Starting from the top left you can see information about the source package, such as versions, and the priority.
(02:21:59 PM) james_w: You also have links to the components of the source package, so using "dget" you can quickly download the current Debian source package.
(02:22:23 PM) james_w: In the right hand pane, you have some links to bug reports for the source package.
(02:22:38 PM) james_w: These include the bug reports on all of the binary packages, but can also reports on the source package itself, so I usually use these so that I don't miss them (though they are rare).
(02:22:58 PM) james_w: Below that you can subscribe to the PTS. This means that if you are really interested in a package you can get emails when things happen with the package.
(02:23:07 PM) james_w: If you select advanced mode from the drop down menu you can select what sort of information you would like to receive.
(02:23:52 PM) james_w: There are loads more links on these pages, I would encourage you to spend some time clicking around in the PTS later, and just see where you end up.
(02:24:17 PM) james_w: It can be hard to remember where to find a bit of information, but most is available somewhere in one of these links.
(02:24:45 PM) james_w: The other important source of information is the Bug Tracking System, known as the BTS.
(02:24:55 PM) nxvl_work: <Kaasethu> QUESTION: //You can quickly go to the page for a package using http://packages.qa.debian.org/package.// -->goes to 404 error
(02:24:58 PM) james_w: If you go to http://bugs.debian.org/src:gnupg you can see the page for the gnupg source package.
(02:26:03 PM) james_w: Kaasethu: that link does as there is no package called "package", try http://packages.qa.debian.org/gnupg instead.
(02:26:39 PM) james_w: Back to the BTS: on the above page you can see an overview of the bugs, with them divided in to sections. You can click on a bug report to get more information.
(02:27:00 PM) james_w: Email is used to communicate with and control the BTS.
(02:27:09 PM) james_w: You can find information on doing this at http://www.debian.org/Bugs/.
(02:27:20 PM) james_w: You can also use "reportbug -B Debian" and the "bts" tool to generate the emails.
(02:28:00 PM) james_w: Some tasks can be a little confusing. If you get stuck try asking in #ubuntu-motu, someone will probably be able to help.
(02:28:24 PM) james_w: So, as eddyMul wanted, we'll now talk about sending patches to Debian.
(02:28:41 PM) james_w: I chose this topic for several reasons.
(02:28:50 PM) james_w: Firstly, it is often requested that Ubuntu forwards more of its patches to Debian.
(02:28:57 PM) james_w: Secondly, it is quite an easy thing to do.
(02:29:10 PM) james_w: Thirdly, if Ubuntu pushes all its patches for a package back to Debian then the package can just be synced in future, reducing the amount of work it takes to maintain them.
(02:29:16 PM) james_w: so, everybody wins.
(02:29:26 PM) james_w: In order to submit a patch to Debian you should first create a standalone patch that fixes a specific issue.
(02:29:35 PM) james_w: You should then check the BTS to see if a bug has been filed about the issue.
(02:29:48 PM) james_w: If it has then you can email the bug report and attach the patch, sending a control command to tag the bug "patch" ("bts tag #12345 patch").
(02:30:11 PM) james_w: The patch can either be a simple patch that they can apply to the unpacked source package.
(02:30:55 PM) james_w: Or, you can create a second source package and use "debdiff" to get the changes, and then filter out anything you don't want to send (perhaps with filterdiff).
(02:31:20 PM) james_w: (debdiff uses interdiff internally, so you can also use that tool directly).
(02:31:43 PM) james_w: In ubuntu-dev-tools there is a command to do exactly that: submittodebian.
(02:32:18 PM) james_w: If you run that then it will generate the debdiff, allow you to edit out parts that you don't want, and then file it as a bug in the BTS.
(02:32:52 PM) james_w: It uses reportbug for the last part, so you can attach the patch to an existing bug as well.
(02:34:10 PM) james_w: eddyMul> QUESTION: " you can email the bug report and attach the patch": how do I email the bug report? do I email the maintainer? (I thought I should be commenting on a bugzilla entry, or something like that....)
(02:34:58 PM) nxvl_work: james_w: would you answer this one?
(02:35:03 PM) james_w: you email the bug itself. You can either compose a mail to 12345@bugs.debian.org to send a message to bug #12345, or you can use "reportbug -B Debian"
(02:35:28 PM) nxvl_work: BTS it's not like LP
(02:35:41 PM) nxvl_work: on LP we just add a comment and attach our patches in there
(02:35:51 PM) nxvl_work: on BTS all the changes are handled by mail
(02:36:10 PM) nxvl_work: you report bugs by mail, change the status by mail, add comments by mail
(02:36:22 PM) nxvl_work: the only way to interact with BTS is by mails
(02:36:29 PM) james_w: I tend to do these steps by hand as I am familiar with the BTS, but if you are not reportbug should be very useful.
(02:36:37 PM) james_w: You can see one of the mails that I sent at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=451683.
(02:36:49 PM) james_w: One thing that we would like you to do when filing bugs in Debian, with or without a patch, is to set a "usertag" on them.
(02:36:49 PM) nxvl_work: you can find more information about this on http://www.debian.org/Bugs/
(02:37:29 PM) james_w: This allows the bugs to be tracked as one, so that it can be seen where Ubuntu is contributing back.
(02:37:36 PM) james_w: You can find out how to do this at https://wiki.ubuntu.com/Bugs/Debian/Usertagging.
(02:37:58 PM) james_w: I would like to point you to one last thing that will make all of this easier in an environment where we have multiple people working the packages.
(02:38:07 PM) james_w: That is the proposal described at https://lists.ubuntu.com/archives/ubuntu-devel/2007-November/024743.html.
(02:38:16 PM) james_w: If you could all start using this technique when creating patches it will make it easier to understand what patches are meant for upstream.
(02:38:39 PM) james_w: My closing comment would be that there are many people involved in Ubuntu development that are knowledgeable about Debian, so if you need to know where to find something, or would like advice about whether a change is applicable to Debian then you should just ask.
(02:39:05 PM) james_w: Now, nxvl_work is going to take over for the rest of the session.
(02:39:12 PM) nxvl_work: james_w: thanks
(02:39:14 PM) nxvl_work: Ok
(02:39:43 PM) nxvl_work: I will talk a little about the best practices on Debian sharing
(02:40:02 PM) nxvl_work: Sometime we want to help, but we do to exact oposite
(02:40:21 PM) nxvl_work: so, please keep in mind some of the things i'm going to tell you
(02:40:59 PM) nxvl_work: There are many ways that you can work together with Debian.
(02:41:20 PM) nxvl_work: For instance you can share patches, work together to package new upstream versions,
(02:41:26 PM) nxvl_work: coordinate transitions, security fixes. You can even become maintainer
(02:41:34 PM) nxvl_work: or co-maintainer of a package in Debian so that the package can always
(02:41:48 PM) nxvl_work: be used unmodified in both with less work.
(02:42:17 PM) nxvl_work: ok, that part is not as simple as it look you need to be careful on what you are doing
(02:42:28 PM) nxvl_work: Many of the bugs on ubuntu are present on debian
(02:43:10 PM) nxvl_work: but it's important to keep in mind and be VERY careful is that no every bug on ubuntu
(02:43:24 PM) nxvl_work: applies to debian, there are software error which are not reproducible on debian
(02:43:31 PM) nxvl_work: so if you will ping the maintainer or report back to debian
(02:43:55 PM) nxvl_work: you must be sure it applies, so you don't make the Debian maintainer work and test looking for a bug that doesn't exist.
(02:44:22 PM) nxvl_work: Also if you are going to send your work to them you must keep in mind that he is also a volunteer as you are
(02:44:45 PM) nxvl_work: they can add your changes if they think it will help them
(02:45:00 PM) nxvl_work: but maybe they don't want to apply them
(02:45:09 PM) nxvl_work: also they don't want to read long diff files with lots of changes
(02:45:19 PM) nxvl_work: it's better to send them small and separate patches with your
(02:45:28 PM) nxvl_work: changes so they can use the ones they find usefull and discards the ones
(02:45:33 PM) nxvl_work: who doesn't
(02:45:39 PM) nxvl_work: Ok, but, what if i'm a triager?
(02:46:02 PM) nxvl_work: You can test if the bugs are also present on debian and report them back using BTS, being carefaul on the points talked earlier
(02:46:35 PM) nxvl_work: also, LP and bts have a "link" option so you can have on the Bug report at LP the link to the debian bug report
(02:46:43 PM) james_w: <tacone> QUESTION: how are we going to know that a bug exists or not in Debian ?
(02:46:47 PM) nxvl_work: so it's easier to keep track of what's going on with the debian bug
(02:47:00 PM) nxvl_work: tacone: easy, try to reproduce it on Debian :D
(02:47:15 PM) nxvl_work: tacone: i have always a Virtual machine running debian for test purposes
(02:47:34 PM) nxvl_work: also you can have a chroot environment
(02:48:04 PM) nxvl_work: ok, back to the LP and BTS syncroniztion
(02:48:10 PM) james_w: there are some packaging bugs that can be checked just by doing things like looking at the dependencies in Debian, but most do need an environment to test.
(02:48:30 PM) james_w: Also, you could ask someone with a Debian system to test.
(02:48:44 PM) nxvl_work: this way it's easier to keep track of what's going on with the debian bug
(02:49:04 PM) nxvl_work: also bts has an option like this that you can use with the command (mail command) 'forwarded $bug $upstream_url'
(02:49:34 PM) nxvl_work: so you can have it backwards, so please use the tools to have a better comunication and avoid the packagers to duplicate efforts on patching.
(02:49:54 PM) nxvl_work: And always keep it mind that Debian makes things LOTS easier to us
(02:50:14 PM) nxvl_work: so we need to try to do our best to make things LOTS of easier for them also
(02:50:15 PM) nxvl_work: :D
(02:50:18 PM) james_w: <eddyMul> QUESTION: does ubuntu track debian unstable? testing?
(02:50:20 PM) nxvl_work: ok, now with the questions
(02:50:32 PM) nxvl_work: eddyMul: track as in syncs from?
(02:50:39 PM) eddyMul: eddyMul: yes
(02:50:55 PM) nxvl_work: unstable
(02:50:59 PM) nxvl_work: always unstable
(02:51:02 PM) nxvl_work: <snhome> QUESTION What am I missing in explanation? If Ubuntu and Debian are so similar why have Ubuntu?
(02:51:14 PM) james_w: though it is possible to sync from elsewhere, e.g. experimental
(02:51:14 PM) nxvl_work: snhome: that's a common but hard question
(02:51:40 PM) nxvl_work: eddyMul: oh yes i forgot, somethimes we sync directly from upstream
(02:51:50 PM) nxvl_work: eddyMul: but commonly from sid (unstable)
(02:51:58 PM) nxvl_work: snhome: ok, on your question
(02:52:21 PM) nxvl_work: snhome: there are some differences between Ubuntu and Debian, and they go on their policies
(02:53:57 PM) nxvl_work: for example, ubuntu install some restricted driver out of the box, debian would NEVER do it
(02:54:01 PM) james_w: Ubuntu changed some things that would have been very difficult or impossible to do in Debian.
(02:54:40 PM) james_w: Also, Ubuntu has a smaller focus in terms of things like supported architectures, meaning it can focus more energy in to the areas it does concentrate on.
(02:55:20 PM) nxvl_work: so Ubuntu and Debian are similar in some ways, but VERY different on others
(02:55:58 PM) james_w: any last questions, we have a couple of minutes left.
(02:56:28 PM) nxvl_work: mathiaz: around?
(02:56:37 PM) seajazz: thanx
(02:56:42 PM) snhome: thanks
(02:56:52 PM) james_w: thanks all.
(02:57:05 PM) james_w: and thanks nxvl.
(02:57:49 PM) nxvl_work: oh yes
(02:58:06 PM) Unksi: thank you :)
(02:58:12 PM) nxvl_work: another difference is that on Ubuntu we like to HUG each other a LOT!