## page was renamed from KubuntuTutorialsDay/Merging
{{{
22:02 < Riddell> next talk is about merging packages
22:03 < jussi01> Riddell: yippee!
22:03 < eagles0513875> Riddell: is there going to be a link to these talks
22:03 < Riddell> in an hour we have a talk about pyqt
22:03 < Riddell> if you want to take part in the pyqt talk please add the hardy ppa archive "deb http://ppa.launchpad.net/kubuntu-members-kde4/ubuntu hardy main"
22:03 < Riddell> to sources.list
22:03 < Riddell> and install libqt4-gui and python-qt4
22:03 < Riddell> but now, who's here for package merging?
22:04 < nosrednaekim> Riddell: what about webkit, though that was separate
22:04 < Sundance> Riddell: Hiya! For those of us on other distros, will our regular installs of PyQt4 do?
22:04 < eagles0513875> Riddell: question for u there was a link to what one needs to bug fix to have installed on the machine but i cant seem to find the page
22:04 < HappySmileMan> Riddell: Isn't it supposed to be plasma development, not just pyqt?
22:04 < Riddell> eagles0513875: ask in the bug fix talk leter
22:05 < Riddell> Sundance: if it's up to date enough
22:05 < Riddell> qt 4.4
22:05 < Riddell> so Merging!
22:05 < eagles0513875> Riddell: i actually im soon heading to bed its quite late here :(
22:05 < Riddell> as you know, software comes in packages
22:05 < Riddell> someone writes the source, we write some packaging
22:05 < Riddell> the packaging has meta data and the rules for compiling the package
22:06 < Riddell> it gets compiled into binaries which is what runs on your computer
22:06 < Riddell> in the ubuntu world we take most of our packages from Debian
22:06 < Riddell> at the start of our 6 month cycle
22:06 < Riddell> then we stabalise them
22:06 < Riddell> and release a distro after 6 months
22:07 < Sundance> Riddell, nosrednaekim: Ack, only got Qt4.3 here. (Well, I've got one hour to see to it, right?)
22:07 < Riddell> often we make improvements to the packages
22:07 < Riddell> newer version or adding bugfixes
22:07 < Riddell> or changes in policy, removing mp3 support for example which we can't put on our CDs
22:08 < Riddell> we try to keep the difference between our packages and debian's packages as small as possible
22:08 < Riddell> because that's less work for us
22:09 < Riddell> so at the start of the development cycle, if there's a package which has been changed in ubuntu
22:09 < Riddell> we grab the current debian package, add in the ubuntu changes again
22:09 < Riddell> and upload
22:09 < Riddell> that way we keep as close to debian as we can
22:09 < Riddell> and if all is working well we send the change to the debian maintainer
22:09 < Riddell> so they get whatever the improvement is if appropriate
22:09 < Riddell> and we don't have to add it any more
22:10 < Riddell> we are now at the start of the intrepid cycle
22:10 < Riddell> and there are lots of packages to be merged with newer debian versions
22:10 < Riddell> http://merges.ubuntu.com/universe.html
22:10 < Riddell> lists all the universe ones
22:10 < Riddell> supert0nes: yes
22:10 < eagles0513875> Riddell: so kinda offtopic but you are the founder of ubuntu
22:10 < Riddell> eagles0513875: no
22:11 < Riddell> main.html has all the main packages that need merging
22:11 < eagles0513875> Riddell: i didnt get what Nightrose said before when she introduced you
22:11 < Riddell> shall we try merging a package?
22:11 < Riddell> grump, netsplit
22:12 < Riddell> this is kubuntu so the packages we're interested in are the kde ones, because we're all kde fans
22:12 < Riddell> I see keurocalc as an easy looking target
22:12 < Riddell> the current ubuntu version is 0.9.6-1ubuntu2
22:12 < Riddell> that means it's keurocalc version 0.9.6
22:12 < Riddell> debian version 1
22:12 < Riddell> we took that debian version and made two uploads of it, so ubuntu version 2
22:13 < Riddell> the debian version is listed as 0.9.7-1
22:13 < Riddell> so that's a new version from the keurocalc developers
22:13 < Riddell> mkdir keurocalc
22:13 < Riddell> cd keurocalc
22:13 < Riddell> mkdir current
22:13 < Riddell> cd current
22:14 < Riddell> we need to get the current ubuntu version
22:14 < Riddell> which we can find on launchpad https://launchpad.net/ubuntu/+source/keurocalc
22:14 < Riddell> clicking on the intrepid version https://launchpad.net/ubuntu/intrepid/+source/keurocalc/0.9.6-1ubuntu2
22:14 < Riddell> and under Download are the three files that make up the source packaging
22:14 < Riddell> so download those three files with wget
22:15 < Riddell> then extract them with  dpkg-source -x *.dsc
22:15 < Riddell> a source package is made up from the upstream developers .tar.gz
22:15 < Riddell> plus a .diff.gz which adds a debian directory with the packaging information
22:15 < Riddell> and the .dsc is a description file
22:16 < Riddell> cd keurocalc-0.9.6/
22:16 < Riddell> less debian/changelog
22:16 < Riddell> the changelog file lists what has happened with each packaging upload
22:16 < Riddell> looking at the most recent merge version
22:16 < Riddell> keurocalc (0.9.6-1ubuntu1) edgy
22:17 < Riddell>   * Merge from debian unstable.
22:17 < Riddell> whoever did that merge was naughty and didn't list what the ubuntu change actually was
22:17 < Riddell> so lesson number 1, always state what you've done to the packaging!
22:17 < Riddell> else it can be very hard to find out in the future
22:18 < Riddell> looking further down we can find what it is
22:18 < Riddell> keurocalc (0.9.4-5ubuntu1) dapper; urgency=low
22:18 < Riddell>   * debian/rules:
22:18 < Riddell>     + Use dh_iconcache .
22:18 < Riddell> this is quite a common change
22:18 < Riddell> gnome in ubuntu has an icon caching feature
22:18 < Riddell> so the icons get added to a database at install time, that's done by dh_iconcache
22:19 < Riddell> but I happen to know that is out of date
22:19 < Riddell> since dh_iconcache has been replaced with dh_icons
22:19 < Riddell> so our task now is to take the debian package
22:19 < Riddell> add the dh_icons change
22:19 < Riddell> check it all compiles
22:19 < Riddell> upload
22:20 < Riddell> and suggest the change to the debian maintainer
22:20 < Riddell> upload to ubuntu!
22:20 < Riddell> so it'll appear in the ubuntu repository
22:20 < Riddell> only ubuntu motu and core-dev can upload to the repository
22:21 < Riddell> so until you become a motu you need to ask someone who is to upload for you
22:21 < Riddell> but don't worry, you still get the karma points in launchpad
22:21 < Riddell> Forky: you'd have asked about it :)
22:21 < Riddell> changelog will list the changes
22:21 < jtechidna> Riddell: Where would be a good place to upload the packages for a motu to get 'em?
22:21 < Riddell> if you're unsure what the change is for
22:22 < Riddell> ask here or #ubuntu-motu or you can ask the person who made the change directly
22:22 < Riddell> jtechidna: revu is good, or any web server
22:22 < Riddell> jtechidna: you can also create a bug and attach to that
22:22 < Riddell> so lets grab the debian package
22:22 < Riddell> http://packages.debian.org/ is a handy site for this
22:23 < Riddell> and at http://packages.debian.org/sid/keurocalc  is our package
22:23 < Riddell> on the right is "Download Source Package"
22:23 < Riddell> so make a new directory and change to that
22:23 < Riddell> and download the three source files
22:23 < Riddell> Forky: you don't have to, it's just if you need somewhere to upload
22:24 < Riddell> wget http://ftp.de.debian.org/debian/pool/main/k/keurocalc/keurocalc_0.9.7-1.dsc http://ftp.de.debian.org/debian/pool/main/k/keurocalc/keurocalc_0.9.7.orig.tar.gz http://ftp.de.debian.org/debian/pool/main/k/keurocalc/keurocalc_0.9.7-1.diff.gz
22:24 < Riddell> dpkg-source -x *dsc
22:24 < Riddell> if you cd keurocalc-0.9.7/
22:24 < Riddell> and ls
22:24 < Riddell> you see the source files
22:24 < Riddell> cd debian; ls
22:24 < Riddell> you see the packaging files
22:25 < Riddell> the rules file has all the instructions for compiling and building the package
22:26 < Riddell> it might help to compare Debian's rules file to the one currently in ubuntu   diff -u rules ../../current/keurocalc-0.9.6/debian/rules
22:26 < Riddell> and I can see where the change is
22:26 < Riddell> +       dh_iconcache
22:27 < Riddell> so I edit debian/rules
22:27 < Riddell> in emacs
22:27 < Riddell> but you can use vi
22:27 < Riddell> I also recommend kate
22:27 < Riddell> add in dh_icons  at the same place as dh_iconcache was previously
22:27 < Riddell> and that's my change
22:27 < Riddell> next I need to add to the changelog
22:27 < Riddell> we have a nifty command for this
22:28 < Riddell> dch -i
22:28 < Riddell> which will fire up an editor with a new changelog entry ready to be added
22:28 < Riddell> the current Debian version is  0.9.7-1
22:28 < Riddell> so we want our version to be  0.9.7-1ubuntu1
22:28 < Riddell> which dch should do for you
22:28 < Riddell> make sure the distro field on the top line is set to "intrepid"
22:29 < Riddell> and set the message to "Merge with Debian: remaining change, add dh_icons to debian/rules"
22:29 < Riddell> make sure your name and e-mail are correct
22:29 < Riddell> save and quit the editor
22:29 < Riddell> next we need to merge in the old changelog entries
22:29 < Riddell> so people can see the history of the package in ubuntu
22:30 < Riddell> emacs -nw debian/changelog  ../current/keurocalc-0.9.6/debian/changelog
22:30 < Riddell> emacs lets me edit both the new and old changelog files at once
22:31 < Riddell> I copy all the existing changelog file and use it to replace the entries from 0.9.6-1 in the new one
22:31 < Riddell> save and exit
22:31 < Riddell> and that's out packaging done
22:31 < Riddell> next we need to test build
22:31 < Riddell> debuild  is the command for this
22:31 < Riddell> run that and it'll tell you if you have all the packages needed for building installed
22:32 < Riddell> if you don't, apt-get install them
22:32 < Riddell> and it should compile away
22:32 < Riddell> once it's done, it'll create a .deb package in the directory above the source
22:32 < Riddell> install that with  dpkg --install *deb  and make sure the application runs
22:33 < Riddell> now we just need to build it as a source package
22:33 < Riddell> which is done with   debuild -S -sa
22:33 < Riddell> -S for source package,   -sa makes it include the .orig.tar.gz in the upload
22:34 < Riddell> and if you are a motu you can upload with dput
22:34 < Riddell> if not, put the files on a web server somewhere
22:34 < Riddell> or on revu which is a testing area for packages that people can review
22:34 < Riddell> and ask an motu to upload
22:34 < Riddell> anyone remember the final step?
22:35 < Riddell> exactly!
22:35 < Riddell> the best way is just to file a bug on bugs.debian.org 
22:35 < Riddell> which is done by sending an e-mail
22:35 < Riddell> be polite and say this adds support for gnome's icon cache
22:36 < Riddell> so that is how to merge a package
22:36 < Riddell> often the changes in ubuntu will be more complex than that
22:36 < Riddell> and ofter the changes in ubuntu are no longer needed
22:36 < Riddell> because the same change has made its way into debian
22:37 < Riddell> in that case you file a bug asking for a sync and subscribe ubuntu-archive
22:37 < Riddell> who will sync it directly from debian
22:37 < Riddell> any questions?
22:38 < Riddell> replacing everything below 0.9.6-1
22:38 < Riddell> _gunni_: yes, from the wiki page
22:38 < Riddell> keurocalc is a real example that I just picked out before the talk
22:39 < Riddell> so if you've been following along at home send me in your package after the talks
22:39 < Riddell> and I'll review it an upload
22:39 < Sundance> Riddell: Binary compatibility in Linux -- or lack thereof -- is a pet peeve of mine. :) Is there a way to go right at the packaging level that will ensure maximal bin compat with other distros & future and past Linux versions?
22:39 < Riddell> and there's plenty more examples on http://merges.ubuntu.com/universe.html and main.html to be done
22:40 < Riddell> Sundance: about the only way to ensure that is to include all the libraries in the packaging
22:40 < Riddell> libstdc++, qt, kde, usualy lots more
22:40 < Riddell> which is what google picasa and earth do
22:40 < Riddell> it's pretty ugly
22:41 < Riddell> it's a big advantage of having source available is that distros can compile it
22:42 < Riddell> Sundance: nice in principle, rarely that useful in practice.  distros have different package formats (deb vs rpm) and different versions of the libraries 
22:42 < Riddell> _gunni_: when you're testing it you just build it for whatever you use
22:42 < Riddell> _gunni_: but there's no need to pass the .deb on to anyone, you just pass on the source .orig.tar.gz, .diff.gz and .dsc
22:42 < Riddell> it'll get uploaded
22:43 < Riddell> and there are a bunch of build servers which sit there waiting for packages to compile
22:43 < Riddell> the build servers will compile it for all architectures and put the result in the repos
22:43 < Riddell> sometimes packages don't compile for an architecture
22:44 < Riddell> it can happen with multimedia packages which often use assembler for optimisation
22:44 < Riddell> in which case it won't get any packages for that architecture
22:45 < Riddell> I should also have said that before merging you should check if there's a new version from the upstream developers
22:45 < Riddell> especially with the kde 4 changover, a number of apps will have kde 4 versions coming out
22:45 < Riddell> and for intrepid, we want the kde 4 version if at all possible
22:46 < Riddell> there's a few here that need packaged ftp://ftp.kde.org/pub/kde/unstable/4.0.80/src/extragear/
22:47 < Riddell> (ask if you want to work on any of those, some are already in process, we don't want to duplicate work)
22:47 < Riddell> in hardy they were packaged with a -kde4 added to their name, that should be dropped for intrepid
22:48 < Riddell> packaging isn't hard, it only takes a couple of hours to get the hang of it, but it takes a lifetime to learn all the tricks and details
22:48 < Riddell> dwidmann: as a desktop it will, if there's no kde 4 version of an app we'll keep the kde 3 one
22:50 < Riddell> 4.1 is still in process in intrepid, it has been blocked for a while on all the main inclusion reports which need to be reviewed by people
22:50 < Riddell> JontheEchidna: if there's no ubuntu modifications it'll sync automatically
22:51 < Riddell> JontheEchidna: it'll only not sync automatically if there's "ubuntuX" in the version number
22:51 < Riddell> JontheEchidna: yes, but as I say often the ubuntu changes have been made in debian
22:51 < Riddell> in which case, a sync is what's needed
22:52 < Riddell> ok, pyqt talk in 10 minutes
22:53 < dwidmann> Riddell: while the topic is packaging, I have a question (but no example to attach it to), what's the best approach for packaging a python app?
22:54 < Riddell> dwidmann: copy an existing package :)
22:54 < Riddell> python apps have various build systems
22:54 < Riddell> sometimes they use ./configure, sometimes cmake, sometimes the native python one
22:54 < Riddell> and sometimes none at all
22:55 < Riddell> you need to edit the debian/rules file to run the appropriate commands
22:55 < Riddell> and install the files into debian/tmp
22:55 < Riddell> then we have a python packaging tool called python-central
22:56 < Riddell> and another similar one called python-support
22:56 < Riddell> python-central seems slightly more popular
22:56 < Riddell> they add the appropriate scripts so that .py files are compiled into .pyc files at install time
22:57 < Riddell> dwidmann: if it's just one file, you can add a line to debian/rules under install to   cp mypythonapp debian/<packagename>/usr/bin
22:58 < Riddell> dwidmann: if you're packaging from scratch you can find a similar package and copy the debian files
22:58 < Riddell> or there's an app dh_make which makes template packaging
22:58 < Riddell> supert0nes: that's what python-central does for you in a package
23:00 < Riddell> we have a mentoring system for people starting doing packaging if you want someone to hold your hand while you get started
23:00 < Riddell> ask here or #ubuntu-motu if you want a mentor
23:00 < Riddell> thanks all for listening, bonus points to the first person who sends me that keurocalc merge
23:01 < Riddell> programming... well guess what tutorial is up next!
}}}