Ubuntu Open Week - Maintaining an Ubuntu Package - Mon, Nov 27, 2006

see also Thursday Session.

10:01   LaserJock       Hello everybody!
10:01   LaserJock       My name is Jordan Mantha and I'm a PhD Chemistry student and 
Ubuntu volunteer.
10:01   LaserJock       In Ubuntu I'm a Universe developer, a part of the Documentation Team, on the Edubuntu Council, and am just generally an Ubuntu-holic.
10:01   LaserJock       Today I want to talk a little about what how we maintain software once it's already in our software repositories (Main, Restricted, Universe, Multiverse)
10:01   LaserJock       First of all, in order to keep the noise level down a bit in here, please also join #ubuntu-classroom-chat and when you want to ask a question just put a "Question:" in front of your question. Thanks.
10:02   LaserJock       To be honest I don't have a full hour of lecturing ready
10:02   LaserJock       and everybody goes "Yay!"
10:03   LaserJock       so I'll start out with some material and then open it up for some Q & A
10:03   LaserJock       I think there are 2 things that are important to keep in mind here:
10:03   LaserJock       1. Ubuntu is intimately connected to Debian
10:03   LaserJock       2. Ubuntu uses Launchpad ( http://launchpad.net ) for virtually all package maintenance
10:03   LaserJock       Let's look at both of those a little more carefully.
10:04   LaserJock       Ubuntu, as most of you probably know, is derived from Debian
10:04   LaserJock       which is one of the oldest Linux distros around
10:04   LaserJock       At the beginning of every Ubuntu development cycle the archive admins update the Ubuntu development repository with the packages that are currently in Debian unstable (Sid).
10:04   LaserJock       If the package was previously changed or modified by Ubuntu it will have ubuntu in the version (alacarte's version in 6.10 is  0.10.1-0ubuntu1 for instance).
10:05   LaserJock       If the package has an ubuntu version then it won't be "synced" automatically, but instead a special script called Merge-o-Matic (MoM) will try to merge the changes as best it can and spit out a report on http://merges.ubuntu.com
10:05   LaserJock       *Every* updated package that previously had Ubuntu changes is checked manually and either merged if the old changes are still needed or synced if they can be dropped. This takes a fair amount of time and accounts for a lot of the maintenance work we do in Ubuntu.
10:05   LaserJock       In Universe our primary goal is the manage (and minimize) the divergence we create from Debian. In Main there is a bit more emphasis on going beyond just managing divergence and really into developing and leading the way in desktop development.
10:06   LaserJock       Because of our intimate connection it is important that we keep at least some track of what's going on in Debian. We do this primarily via the Debian Bug Tracking System (BTS)
10:06   LaserJock       http://bugs.debian.org
10:06   LaserJock       and Package Tracking System (PTS)
10:06   LaserJock       http://packages.qa.debian.org
10:06   LaserJock       This allows us to keep track of Debian versions and bug fixes.
10:07   LaserJock       OK, so now I'll give you a little time to digest all that and ask any questions so far
10:07   LaserJock       nice, I see I've explained everything perfectly :-)

[tictacaddict] Can unmodified debian packages normally be installed in Ubuntu? will there be problems with dependencies sometimes?

• yes,there are differences although the source packages may not be different, every package in Ubuntu is rebuilt in an Ubuntu environment so the resulting binary packages often have slightly different dependecies

[whowe] Will Ubuntu packages work fine in Mepis or should you use the Debian packages, I have noticed on a machine with Mepis that it uses Ubuntu repositories

• they might, but there certainly isn't any way to say for sure. It's obviously best to use packages built for your distro

[danbuntu,cbx33] By roughly how much do the packages change, what changes and why?

• primarily the changes are in dependences or if there is something that Ubuntu is pushing forward with examples have been when we used a newer default Python and gcc version we had to "tweak" the dependecies to use those versions. Some package change very little, like literally one line other are pretty heavily patched it depends on what Ubuntu wants to have to maintain because the next time around we are going to have to merge those changes back in

[greguti] how many people spend their time checking the MoM packages? You said it takes a lot of time, but for how many developpers?

• well, generally it takes all the developers a few months to get the process comleted. Main has more paid developers and there is a larger packages/devs ratio, so it goes faster. To give you an idea, there are 18656 source packages in Edgy Universe, 1250 of them have ubuntu versions and have to be merges/synced. In Main the number is 5382 and 978 so total in Ubuntu there are roughly 2000 packages that have to be manually looked at with each release and there are roughly 80-100 people doing it. but not all of them are updated every release cycle. Not all of them are updated every release cycle, but each package must be looked at, modified if needed, and rebuilt and tested befor it ever gets uploaded

[La_PaRCa] Is the tracking of the debian BTS and PTS automatic for each package or is it "by hand"?

• some of both

[cbx33] Does one need an intimate knowledge of make/automake to be able to package "make" source tree?

• intimate, no, some is helpful though

[tenshu] Is debian merging changes with ubuntu packages the same way ubuntu does?

• no, Debian has a different maintainance structure than Ubuntu. In Debian each package has a maintianee or even team of maintainers that "own" a package. In Ubuntu we use team maintianence. So MOTU ( Masters of the Universe) maintain *all* of the Universe repo. An example of a semi-automatic system is a list I maintain for the MOTU Science team, http://tiber.tauware.de/~laserjock/motuscience/feisty/all.html is an example of a system we set up to track Ubuntu and Debian changes

[adefelice] Is it possible that a package which has no bugs in debian has bugs in Ubuntu?

• yes it is possible, anytime you modify something there is a chance you introduce a bug. we'd like to think we fix more then we introduce

[lotusleaf] Are there any plans to keep WINE up-to-date and available as a .deb within the official Ubuntu repos, rather than have it sit at the version it is now within them, leaving people to use a 3rd party repo without a gpg key (unless they compile from source) to get the latest version?

• well, that is always the goal. we don't sit around all day thinking how we can give users old, crusty software. we try to do our best to have as stable and updated of software as we can, but wine is maintained by the Ubuntu community and it's a tough packages. so you are more than welcome to help maintain it and we can help you along the way

[orphean] what's the recommended route for some interesting in contributing to package maintaing?

• Get in contact with the MOTU team either #ubuntu-motu or the ubuntu-motu mailing list on lists.ubuntu.com. MOTU does a lot of work on helping people learn to create and maintain packages. the pacakging guide is great too

10:26   LaserJock       the second thing I wanted to talk about was Launchpad
10:27   LaserJock       The first thing is you have to figure out how to use Launchpad. It is a rather large and sometimes confusing system but it also houses a lot of power.
10:27   LaserJock       My primary advice to people who want to use Launchpad very much is to learn how to create your own URL.
10:27   LaserJock       For instance, if you want to know about a particular source package use:
10:27   LaserJock       http://launchpad.net/distros/ubuntu/+source/<packagename>
10:27   LaserJock       If you want to see all the bugs for a package just add on +bugs:
10:27   LaserJock       http://launchpad.net/distros/ubuntu/+source/<packagename>/+bugs
10:27   LaserJock       Parts of the URL with the + in front are important, they are like modifiers to the thing that goes before it. In this case we want to see bugs for <packagename>
10:28   LaserJock       so really package maintainance in Ubuntu is broken down into 2 parts for the most part
10:28   LaserJock       the first is the merge/synce process that we've already talked about
10:28   LaserJock       where we sort of take a new snapshot of Debian
10:29   LaserJock       the second part comes after we've done that and we've "frozen"
10:29   LaserJock       and that is bug fixing
10:30   LaserJock       for that we use the Launchpad bug tracking system called Malone

[danbuntu] there's always talk of weather rpm, tqz, deb or what ever is better. Do you think that the current deb system is still relevent and suitable?

• well, that is an interesting question. my answer is I haven't seen anything better that can be used at this scale. I think debs are still very relevent and suitable. there are a major reason why Debian is as stable and secure as it is and it is also in development

<tictacaddict> Malone like Mal-Won or Maloney

• ma-lone is kind of how i say it

<tenshu> It seems to have a lot of work done with merging/syncing; But why does it take so long to get a package accepted through REVU?

• ah, good question. the basic answer is (in my experience) it is very difficult to review other people's packages. in fact I can often take more time reviewing a package than the person did actually making it. it's also a time managment issue. maintaining the packages we already have takes a lot of time too. people want new packages. people want the latest packages. people want bug-free packages. At some point we can't do it all :-). so we try to work on a little bit of all of those. and it is only 6 month to do this all ;-). we spend a few months getting all synced up to Debian. we spend a fair amount of time bug fixing. people intersted in the timing should check out http://wiki.ubuntu.com/FeistyReleaseSchedule

<somerville32> Can you explain backports and stable release updates? How easy is it to get each approved respectively? What circumstances call for these to occur?, etc.

• https://wiki.ubuntu.com/StableReleaseUpdates https://help.ubuntu.com/community/UbuntuBackports Once a Release has been released it is really frozen. We don't add any totally new packages and we have 3 channels for updates

  • -security (what the name suggests, security fixes) -updates (major and high impact bug fixes. "Ma, my computer ate my homework" -backports ( taking packages from the development release and building them for a stable release)
  we have policies in place (as Lure gave some URLs for) for all of these

<greguti> why did you choose to use Launchpad and not some other bug-tracking and versioning system? What are the benefits of this system?

• ah, well Launchpad is written by Canonical. we used to use bugzilla for our bug-tracking but Canonical wanted to build a large infrastructure for distro and software development so we have bug tracking, translations, specification tracking, meeting tracking, teams so when Launchpad and Malone seemed usable we switched over and now we are using it for package building and archiving

<cbx33> Are there plans to make the REVU process documented a little clearer?

• of course we have plans of making everything perfectly documented and running like a well-oiled-machines. however, that takes a lot of manpower and will take some time

<kudzubane> are there formal regression tests for new/updated packages during the revu process?

• REVU is primarly designed for brand new packages, ones that don't exist in Ubuntu/Debian currently we usually use bug reports for patches to existing packages. there are plans afoot for distro-wide testing. I can't speak a whole lot about Main on this but there are plans. so far it's more-or-less been up to individual developers to test things before uploading

<diocles> Where should bugs against feisty get filed in launchpad? Do we have to check whether the package differs to the one in Edgy?

file it like any bug. it's always helpful to say what release you are running and what version of the package you are using

<danbuntu> do you ever get time to sleep or is all just package, package, package?

• what's sleep? LaserJock never sleeps, he just keeps on going

<gumpa> what does the final '4' mean in the package version: 0.9.22-0ubuntu4

• well, the first time we changed it we used 0.9.22-0ubuntu1. the 4 just means we've updated that package 3 more times, since then

<Terminus> Wouldn't it be better to have some of the newer packages make it into -updates instead of -backports? Sometimes people don't like enabling -backports. Maybe -updates would be appropriate? Example, the recent changes upstream for flashplugin-nonfree. It meant that flash was broken on newly installed dapper systems until you use -backport or manually install the -backport deb.

• well, that's a tough question. we have worked out a Stable Release Updtate policy that should help clear up what does and doesn't go into updates and makes sure it is properly tested. the problem is that -updates and -backports have a different focus.
  • -updates is focused at fixing bugs in an existing package that has to make sure that we aren't introducing new bugs. we want to make the user's system *more* stable not less -backports is focused more at getting the latest versions
  I'm not sure about flash but it does take more time to get into -updates then -backports because of the stability issue

<tiagoboldt> How do we 'get our hands dirty'?

• I'd really encourage everybody interested in contributing to check out the MOTU http://wiki.ubuntu.com/MOTU and #ubuntu-motu these are the community volunteers that make Universe and Multiverse work and they are the entry point into learning how to package and maintianing packages in Ubuntu.

10:53   LaserJock       I can in now way do justice to the topic in 1 hr
10:54   LaserJock       but hopefully I've given you a few things to chew on and perhaps answered a few of your questions
10:54   LaserJock       we really like to emphasize community participation
10:54   LaserJock       and you are really welcome to help us out, no matter what your skill level is
10:55   LaserJock       we aren't just looking for programmers (although they are handy too ;-) )
10:56   LaserJock       it's the system we wrote to allow us to review and include packages from the community into Universe

<zi99y> Wouldnt it make sense to have the auomatix packages available as standard but restricted depending on your location. i.e. if they are legal in your part of the world?

• no. Automatix is something that is used quite a bit in forums community but it is essentially obsolete and I really don't see it serving a purpose anymore once Feisty is released. it filled a gap for a while, but it's probably time for it to go soon but that's just my opinion see also https://wiki.ubuntu.com/CommonCustomizations. we saw lots of breakage in the Dapper->Edgy upgrades because of third party repos and scripts like Automatix. it's just hard on the system to do things like that

<tm|ubuntu> Is there any plan to have something analogous to debian-volatile?

• well, there has been a longstanding proposal for something called Grumpy Groundhog. it might be somewhat similar, although more towards debian's experimental repo

<_MMA_> Will the "becoming a MOTU" get a structure? ie: Step1, Step2 and so on?

• yes it will

11:00   LaserJock       ok, I'm done
11:00   LaserJock       Thanks everybody