Ubuntu Open Week - Maintaining an Ubuntu Package - Thu, Nov 30, 2006
see also Monday Session.
=== ..[topic/#ubuntu-classroom:LjL] : Welcome to Ubuntu Open Week, Nov 27 - Dec 2 between 3-10pm UTC, to see this in your timezone, visit http://tinyurl.com/ykqc67 | Schedule: https://wiki.ubuntu.com/UbuntuOpenWeek | Friday's Freshers' Day! Join #ubuntu-freshers for talk+questions | /msg ubotu lots for logs | Please keep support questions in #ubuntu | Class discussions+questions in #ubuntu-classroom-chat | Current Session: Maintaining an Ubuntu package 09:01 LaserJock ok, time to get started? 09:02 LaserJock Hello everybody!My name is Jordan Mantha and I'm a PhD Chemistry student and Ubuntu volunteer. 09:02 LaserJock In Ubuntu I'm a Universe developer, a part of the Documentation Team, on the Edubuntu Council, and am just generally an Ubuntu-holic. 09:02 LaserJock Today I want to talk a little about what how we maintain software once it's already in our software repositories (Main, Restricted, Universe, Multiverse) 09:03 LaserJock so far during the Open Week we've had talks on how to package 09:03 LaserJock how to patch packages 09:03 LaserJock etc. 09:04 LaserJock so I'm not really going to cover that so much 09:04 LaserJock I think there are 3 things that are important to keep in mind here: 09:04 LaserJock 1. Ubuntu is intimately connected to Debian 09:04 LaserJock 3. Ubuntu relies on thousands of "upstreams" 09:04 LaserJock 2. Ubuntu uses Launchpad ( http://launchpad.net ) for virtually all package maintenance 09:05 LaserJock Ubuntu is a Linux distro 09:05 LaserJock which among other things, means it usually doesn't write it's own software 09:05 LaserJock but instead collects other people's software (upstreams) and "glues" them together and makes sure they work right 09:06 LaserJock so lets look at the 3 things I've outlined above in a little more detail 09:06 LaserJock 1. Ubuntu is intimately connected to Debian 09:06 LaserJock Ubuntu is a Debian-based distro 09:06 LaserJock that's probably not surprising to anybody 09:07 LaserJock but we don't fork Debian 09:07 LaserJock instead at the beginning of each development cycle we take a snapshot of Debian's unstable repo 09:08 LaserJock and we spend much of the first half of the development release cycle merging those changes back into Ubuntu 09:09 LaserJock ok, so the question is how do we keep track of what we've changed and what's straigh from Debian 09:09 LaserJock to differentiate we use a particular versioning scheme 09:09 LaserJock any source package that has been changed or modified in any way will give a ubuntuX version put on the end 09:10 LaserJock where X starts at 1 and just increments with each upload we make that keeps the same base Debian version 09:10 LaserJock so when we go through our merging/syncing phase at the beginning of the cycle we determine which packages were changed in the previous version 09:11 LaserJock and if they weren't changed, we just sync the Debian package straight over 09:11 LaserJock however, if there was a change a handy script called Merge-o-Matic (MoM) will attempt to merge the package in a semi automated way 09:12 LaserJock the thing to note here is that *every* package that had a previous Ubuntu change has to be manually checked before it is either synced (if Debian picked up our changes and we don't need them anymore) or merged (we still need them) 09:13 LaserJock so just to see the scale we are taking about here are some stats for edgy: 09:13 LaserJock In the Main repo there are a total of 5382 source package 09:14 LaserJock 978 of them have ubuntuX versions and have to be manually checked 09:14 LaserJock in Universe there are 18656 source packages 09:14 LaserJock 1250 of them have ubuntuX versions 09:15 LaserJock so that's over 2000 packages that have to be manually checked, merged, built, and tested before being uploaded 09:15 LaserJock we have probably around 100 people doing this work 09:15 LaserJock around 80% are volunteer community members 09:16 LaserJock ok, let's take a question break
< rmjb> If every package with an Ubuntu change needs to be manually checked, what does MoM do?
- MoM can make the process easier. what you are looking for in a merge is what Ubuntu changes exist and why they are there. so you have to look at the difference between the Ubuntu package and the Debian package it was based on. you also have to look at what has changed since then in Debian and then go back and make sure you only add in the Ubuntu changes to the new Debian version
< andresmujica> Does it means that Debian takes changes made here at ubuntu for their own packages???
- yes, that is correct. we send all the differences we make to Debian and the Debian maintainers are free to take our changes. we try to maximize cooperation and minimize divergence
[bhale] as an example, Debian and Ubuntu mono teams share many members, and changes flow in 2 directions. Patches are posted here in an automated fashion as well: http://patches.ubuntu.com/ and also on the individual package pages on packages.qa.debian.org
< ailean> can you point us in the direction of some tutorials?
- yes, there is an Ubuntu Packaging Guide. it is shipped in both the Ubuntu and Kubuntu help systems and can also be found on help.ubuntu.com. also check out the Debian documentation at www.debian.org/devel/
[ubotu] The packaging guide is at http://doc.ubuntu.com/ubuntu/packagingguide/C/index.html Other developer resources are at https://wiki.ubuntu.com/DeveloperResources
< urbnsr> Are there packages that start for ubuntu only or do all packages have to come via Debian?
- yes, some packages start in Ubuntu. some start completely elsewhere. the vast majority come from Debian though
[bhale] no, Ubuntu can get new packages, see http://wiki.ubuntu.com/REVU for submitting new ones
< andresmujica> It seems that there' s some work duplicated at several levels, package creator, debian, and ubuntu.. any proposal or way to improve this? or is defacto way?
- well, in essence what we try to do in Ubuntu is take what Debian maintainers do and add on top of it. not repeat it. so if there's a bug that Debian should know about we forward it. if we patch something that is useful for Debian we try to get it to them. so the idea is we are enhancing rather then duplicating Debian's work. there are efforts like utnubu (ubuntu backwords) to help make sure Ubuntu changes get back into Debian, but mostly just good communication
09:24 LaserJock OK, so the second item: Ubuntu relies on thousands of "upstreams" 09:25 LaserJock so Debian is our first upstream most of the time 09:25 LaserJock but they have upstreams too 09:25 LaserJock the original software authors 09:26 LaserJock and if we want our work to help the maximum number of people we need to get our changes to them 09:26 LaserJock also, as software develops, it doesn't do so on the same time line 09:26 LaserJock so we have to be aware of thousands of different projects all releasing at different times 09:27 LaserJock and we have to work to make sure we can produce the most stable and usable distro we can 09:27 LaserJock that really leads to my 3 point: Ubuntu uses Launchpad ( http://launchpad.net ) for virtually all package maintenance 09:28 LaserJock Launchpad is essentally a very large distro managment web app 09:28 LaserJock developed by Canonical (the Ubuntu sponsor company) 09:29 LaserJock and it includes a bug tracker (Malone), specification tracker, translation system, support tracker, etc. 09:29 LaserJock it also houses our archive maintainence and build structure 09:30 LaserJock Launchpad can sometimes be a bit confusing to use when you first get started with it 09:30 LaserJock so here's my #1 tip, learn how Launchpad uses URLs 09:30 LaserJock so you can "build" your own URL to go where you want 09:31 LaserJock it makes finding things pretty easy 09:31 LaserJock For instance, if you want to know about a particular source package use: 09:31 LaserJock http://launchpad.net/distros/ubuntu/+source/<packagename> 09:31 LaserJock If you want to see all the bugs for a package just add on +bugs: 09:31 LaserJock http://launchpad.net/distros/ubuntu/+source/<packagename>/+bugs 09:31 LaserJock Parts of the URL with the + in front are important, they are like modifiers to the thing that goes before it. In this case we want to see bugs for <packagename> 09:32 LaserJock now one of the things that makes Launchpad good for package maintainance and communication with upstream is you can link to other bug trackers 09:32 LaserJock for instance, right now, Launchpad can link and track bugs in Debian, Gnome, KDE, Mozilla to I believe 09:33 LaserJock so if somebody files a bug in Launchpad for Ubuntu 09:33 LaserJock and I see that Debian has reported the same bug 09:33 LaserJock I can link to that one and watch it's progress 09:33 LaserJock and be notified when it's been fixed so that I can make sure to get those fixes
< proppy> sometimes when we package for multi distribution and there is different dependencies, we supply multi control files for each distribution (for example control.dapper which differ from control.sarge), i believe this is not the correct way to do it?
- really what we'd rather see is that difference in versioning and in a revision control. another project that Canonical is supporting is the bazaar revision control system. Launchpad now supports bazaar so that each Launchpad user can keep different "branches" of what they are working on. you can even use a Launchpad team to control access so rather then individual control files. I'd rather see a dapper branch or a sarge branch on bazaar.launchpad.net [bhale] another choice for cooperating with debian developers is svn for your team on alioth.debian.org and make branches off of the trunk
< gumpa> is Launchpad a Zope3 app?
- [ajmitch] yes, it is
< andresmujica> About upstream bug trackers, should i search for a similar bug or report a knew one pointing to the bug reported.. <proppy> Given that i know which dependencies differs from the debian packages, is there a way to provide theses information *in* the debian packages, to they get applied when synced from the unstable repository ?09:41 LaserJock ok, that's a basic overview of what maintaining a package in Ubuntu involves
09:41 LaserJock other then the nitty gritty details of the actual packaging
09:41 LaserJock so I'd like to open it up for any general package maintainance questions
- [bhale] you can make a file called control.in. that dynamically builds a control file, similar to auto* check the Debian Developers Guide for that. you might want to check out #ubuntu-motu so you want to make it so Ubuntu can sync even though there are some slight dependency differences?
[proppy] we have access to debian unstable repository and locally compile and package for ubuntu so i want to make the sync process smooth
- in essence then maintaining it in Ubuntu would be filing a bug report with a debdiff, worst case. if you are actively maintaining the package in Debian and want to make Ubuntu changes, you can contribute them to the archive via a MOTU (or becoming one yourself)
< rmjb> how important is the maintainter's relationship with the upstream developer? Do you need their permission before packaging thier software for Ubuntu?
interesting question. If it's free software then you don't *have* to get there permission, but it's rather bad form to go putting somebodies work in a distro that will go to a few million people and not tell them
in my experience though upstreams are pretty excited to get their work in a distro and usually will work with you pretty well. I maintain 2 packages in Debian. and I talk with both of the upstreams fairly often. and they help me by making any changes necessary to the build structure to work with debian packaging [rmjb] I guess the least you can do it try to inform them
- [bhale] it would be polite to do so
< andresmujica> Is any kind of public building machine or should i compile at my own server and then distribute the package?
- well, the answer is, not quite yet. I know that Canonical is working on building personal package archives into Launchpad that will allow anybody to upload and build packages in the own personal repo space, but that doesn't exist right now. however, getting your package into Ubuntu does have that advantage of building your package on all the supported arhes [bhale] at this time what you want is pbuilder. docs on the wiki
< cbx33> Where can I get more information about maintaining packages using make? How much make knowledge is needed?
- well, of course you can read up on autotools stuff, but in essence you don't really need to know all the details to package but the more knowledge you have the better off you'll be but it's not uncommon for programs to not use make at all. for instance a lot of python packages use distutils etc.
< LjL> Would that, once it exists, also allow building packages without actually having all the required -dev dependencies installed on the local machine, and/or setting up a chroot environment, and similar relatively onerous undertakings?
- indeed it would, although it would be much better to test your packages before hand. we are getting pbuilder docs better and better so hopefully it will be less onerous [bhale] IMO the build service is not meant for testing the build of your package but building and publishing a 'canonical' copy once youve done the work
< somerville32> Can you compare and constrast Stable Update Releases and Backports?
- Stable Release Updates are for very important bug fixes to existing versions
[bhale] https://wiki.ubuntu.com/StableReleaseUpdates. STU refers to $distr-updates, critical bug fixes
- Bugs which may, under realistic circumstances, directly cause a security vulnerability Bugs which represent severe regressions from the previous release of Ubuntu Bugs which may, under realistic circumstances, directly cause a loss of user data
< apokryphos> with regard to Canonical making a build service, have they not considered using the already active/popular SUSE build service?
- not sure, but the idea would be to combine bazaar, personal package archives, etc. all withing one system. [bhale] no, the Canonical service was in works before the SusE service was announced
[somerville32] How do you gauge severity?
- that is up to the people approving updates, but generally you know a severe bug when you see it i.e. segfaults
09:58 LaserJock NOTE: the MOTU (universe maintainers) rock hard core so if you have any further questions pop on over to #ubuntu-motu 09:58 LaserJock or on the ubuntu-motu mailing list on lists.ubuntu.com 09:58 LaserJock thanks bhale