Ubuntu Wiki

Dev Week -- Packaging like a Ninja -- shadeslayer -- Tue, Jul 13th, 2010

(11:54:59 AM) shadeslayer: OK.. so first up,the stuff that takes some time
(11:55:01 AM) shadeslayer: Build Env : https://wiki.kubuntu.org/Kubuntu/Ninjas/BuildEnvironment
(11:55:21 AM) shadeslayer: Read that once and have the build environment ready :D
(11:55:34 AM) shadeslayer: Every few weeks KDE make a new release of their software compilation
(11:55:53 AM) shadeslayer: and our crack team of packaging ninjas jumps into action to package this
(11:57:29 AM) shadeslayer: Please branch bzr branch lp:kubuntu-dev-tools
(11:57:35 AM) shadeslayer: those are the latest tools
(11:58:29 AM) shadeslayer: mhall119: yes
(11:59:42 AM) shadeslayer: First we build all the packages for maverick and these are backported to lucid
(12:00:04 PM) shadeslayer: tech2077: currently the kubuntu-dev-tools are broken
(12:00:44 PM) shadeslayer: bzr branch lp:~kubuntu-members/kubuntu-dev-tools/trunk
(12:00:49 PM) shadeslayer: the right dev tools
(12:02:09 PM) shadeslayer: General Dep Graph : https://wiki.kubuntu.org/Kubuntu/Ninjas/DependencyGraph : is the genral dep graph ninjas follow in their quest to get the latest KDE releases building
(12:02:36 PM) shadeslayer: so first up we have to upload kdelibs and then go upwards onto kdebase and other such stuff
(12:03:18 PM) shadeslayer: KDE released 4.4.92 sources a few days back,so i will be teaching you how to package kdetoys 4.4.92
(12:03:46 PM) shadeslayer: We usually get the tarballs a few days prior to the release so that we get the time to package them and test out any build failiures
(12:03:52 PM) shadeslayer: *failiures
(12:04:21 PM) shadeslayer: http://people.ubuntu.com/~rohangarg/kdetoys-4.4.92.tar.bz2 << Not so secret tar
(12:04:25 PM) shadeslayer: ;)
(12:05:12 PM) shadeslayer: download that tar in a seprate folder
(12:05:17 PM) shadeslayer: lets name it tmp/
(12:05:38 PM) shadeslayer: so , mkdir tmp ; cd tmp ; wget http://people.ubuntu.com/~rohangarg/kdetoys-4.4.92.tar.bz2
(12:06:00 PM) shadeslayer: everyone done?
(12:06:32 PM) shadeslayer: tar -xjvf kdetoys-4.4.92.tar.bz2 comes next
(12:07:18 PM) shadeslayer: <joaopinto> QUESTION: Assuming there are no major build related changes couldn't the entire build-on-new-release process be automated ? Do you really need to manually package it for an inicial upstream build ?
(12:07:57 PM) shadeslayer: joaopinto: yes,but there are huge issues with Beta 1 releases of KDE,loads of missing stuff in install files and deps that need to be added
(12:08:08 PM) shadeslayer: these have to be checked manually and cannot be automated
(12:09:01 PM) shadeslayer: ok now most of our packaging is hosted on bzr
(12:09:11 PM) shadeslayer: so : bzr branch lp:~kubuntu-members/kdetoys/ubuntu -r 60
(12:09:17 PM) shadeslayer: that checks out revision 60
(12:09:45 PM) shadeslayer: done?
(12:10:37 PM) shadeslayer: Also,if your not a k/ubuntu member,like i was when i joined the ninjas,you can push the packaging to your own bzr branch and ask for a merge with that branch
(12:10:52 PM) ClassBot: Mindflyer91 asked: We have to checkout in the temp folder?
(12:10:58 PM) shadeslayer: Mindflyer91: yes
(12:11:23 PM) shadeslayer: that will create 2 folders,one containing the kdetoys sources and the other has the ubuntu/debian dir
(12:11:30 PM) shadeslayer: which has most of our packging
(12:11:34 PM) shadeslayer: *packaging
(12:11:52 PM) shadeslayer: The good thing is,most of the dirty work ends in the .90 releases
(12:12:15 PM) shadeslayer: thats when no more install files need to be edited to make the package work
(12:12:23 PM) shadeslayer: only small alterations are required
(12:13:17 PM) shadeslayer: so,after you have the packaging branched, just copy the debian/ folder over to the extracted kdetoys sources
(12:13:29 PM) shadeslayer: done?
(12:13:49 PM) pjarnahom is now known as pjarnahomzz
(12:14:04 PM) shadeslayer: abhi_nav: just cp -r ubuntu/debian kdetoys-4.4.92/
(12:14:34 PM) shadeslayer: copy the debian folder from the ubuntu folder into the kdetoys-4.4.92 folder
(12:14:51 PM) shadeslayer: everyone done?
(12:15:11 PM) shadeslayer: ok so far so good :D
(12:15:25 PM) shadeslayer: now, cd kdetoys-4.4.92
(12:15:29 PM) shadeslayer: dch -i
(12:16:02 PM) shadeslayer: at the top,edit the version to 4:4.4.92-0ubuntu1~ppa1
(12:16:25 PM) shadeslayer: if your on maverick and 4:4.4.92-0ubuntu1~lucid1~ppa1 if you are on lucid
(12:16:56 PM) shadeslayer: abhi_nav: debchange <<
(12:17:49 PM) shadeslayer: then after the * add : New upstream release
(12:17:55 PM) shadeslayer: this marks the changelog
(12:18:03 PM) shadeslayer: that a new upstream KDE version was released
(12:19:25 PM) shadeslayer: done?
(12:19:30 PM) shadeslayer: should i move forward? :D
(12:20:09 PM) shadeslayer: ok,so moving forward
(12:20:19 PM) shadeslayer: now open : nano debian/control
(12:21:12 PM) shadeslayer: this is the most important file apart from our rules file that helps in packaging
(12:21:32 PM) shadeslayer: as you can see it describes each package,what it does,and what needs to be pulled in to make it build :D
(12:21:50 PM) shadeslayer: in line 7 you will see something like kde-sc-dev-latest (>= 4:4.4.90)
(12:22:27 PM) shadeslayer: kde-sc-dev-latest is a meta package that pulls in other dependencies to make the package work
(12:22:48 PM) shadeslayer: now change that 4:4.4.90 to 4:4.4.92
(12:23:06 PM) shadeslayer: since kde released a new version and new deps are needed ;)
(12:23:14 PM) shadeslayer: done?
(12:23:41 PM) ***shadeslayer hands orange ninja belt to tech2077 chilicuil and abhi_nav_
(12:23:52 PM) shadeslayer: congrats new ninjas :D
(12:24:06 PM) shadeslayer: ok now thats all we need to edit in that file :D
(12:24:13 PM) shadeslayer: close with Ctrl+X and y
(12:25:05 PM) shadeslayer: ok,now thats all that needs to be done in that package
(12:25:06 PM) shadeslayer: :D
(12:25:35 PM) shadeslayer: just edit debian/changelog and add - Bump kde-sc-dev-latest to 4.4.92 below the *
(12:25:40 PM) shadeslayer: something like http://bazaar.launchpad.net/~kubuntu-members/kdetoys/ubuntu/annotate/head:/debian/changelog
(12:26:12 PM) shadeslayer: everything done?
(12:26:49 PM) shadeslayer: good!
(12:26:57 PM) shadeslayer: now,lets start building!!!!!!!
(12:27:11 PM) shadeslayer: in the kdetoys dir,just run : pdebuild
(12:27:20 PM) shadeslayer: thats it!
(12:27:39 PM) shadeslayer: ( supposing you have pbuilder and some ninja hooks ;) )
(12:28:10 PM) shadeslayer: my favourite hooks are B10list-missing  C10shell
(12:28:30 PM) shadeslayer: the B10list-missing hook prints out a list of missing files at the end of the build
(12:28:45 PM) shadeslayer: pretty useful when your packaging the first beta release of KDE
(12:29:15 PM) shadeslayer: the C10shell is another one which drops to a shell and install vim for me to inspect the problem
(12:29:51 PM) ClassBot: simar asked: How this different from MOTO in ubuntu ?? I look for analogous names of ninja as i'm more familiar with ubutnu ??
(12:30:02 PM) shadeslayer: MOTU == Masters of the Universe
(12:30:25 PM) shadeslayer: these people are responsible for the universe section of the archives
(12:30:27 PM) shadeslayer: !motu
(12:30:39 PM) ***shadeslayer looks for ubottu
(12:31:20 PM) shadeslayer: KDE packages are in the main section of the archives ( most of them that is ;) )
(12:31:35 PM) shadeslayer: so packaging KDE and MOTU are 2 different things
(12:31:47 PM) ClassBot: joaopinto asked: regarding kde-sc-dev-latest, how do you know that the new version is required to build this particular package ?
(12:31:52 PM) shadeslayer: joaopinto: good question
(12:32:02 PM) shadeslayer: kde-sc-dev-latest is a meta package
(12:32:21 PM) shadeslayer: it depends on other packages,but does not install anything by itself
(12:32:49 PM) shadeslayer: also,in order to build kdetoys 4.4.92,you will need kdebase 4.4.92 , this is hard coded in the cmake files
(12:33:29 PM) ClassBot: simar asked: I have learnt some packaging skills yesterday from danial also but after then I don't know what to do and where and frm where to start ?? How to get in team .. Could you please tell me a bit abt dat here also ..
(12:33:50 PM) shadeslayer: simar: we idle in #kubuntu-devel,poke us there and we will hand you work :D
(12:34:09 PM) ClassBot: tech2077 asked: I can't build this myself, i seem to not have a lot of the dependencies
(12:34:19 PM) shadeslayer: tech2077: can you pastebin this error?
(12:34:30 PM) shadeslayer: also which version are you on?
(12:34:47 PM) ***shadeslayer checks time
(12:34:53 PM) shadeslayer: oh.. 30 mins more :D
(12:35:25 PM) abhijain is now known as yoku
(12:35:43 PM) warp10 is now known as warp10-captato
(12:35:44 PM) shadeslayer: tech2077: ok exit pbuilder if it hasnt already
(12:36:44 PM) shadeslayer: tech2077: now run : sudo pbuilder --login  --save-after-login
(12:36:55 PM) mode (+v Quintasan) by ChanServ
(12:37:21 PM) shadeslayer: everyone welcome Quintasan
(12:37:26 PM) Quintasan: \o
(12:37:28 PM) shadeslayer: another ninja
(12:37:41 PM) shadeslayer: dpkg-checkbuilddeps: Unmet build dependencies: kde-sc-dev-latest (>= 4:4.4.92) cmake pkg-kde-tools (>= 0.6.4) kdebase-workspace-dev (>= 4:4.4) libphonon-dev (>> 4:4.7.0really) libstreamanalyzer-dev (>= 0.6.3) libqimageblitz-dev
(12:37:52 PM) shadeslayer: that says that you do not have the proper build deps
(12:38:09 PM) shadeslayer: Quintasan: this will be over in 10 mins,we can take on neon then :D
(12:38:19 PM) shadeslayer: so everyone run sudo pbuilder --login  --save-after-login
(12:38:47 PM) shadeslayer: then : add-apt-repository ppa:kubuntu-ppa/beta
(12:38:56 PM) shadeslayer: done?
(12:40:07 PM) ***shadeslayer pokes all orange belt ninjas will pointy sword
(12:40:12 PM) Mitch is now known as Guest9138
(12:40:21 PM) yoku is now known as abhijain
(12:40:39 PM) shadeslayer: ok then : apt-get install nano
(12:40:53 PM) shadeslayer: pbuilder doesnt have a editor by default
(12:41:20 PM) shadeslayer: then : nano /etc/apt/sources.list
(12:41:24 PM) shadeslayer: and add : deb http://ppa.launchpad.net/kubuntu-ppa/beta/ubuntu lucid main
(12:41:30 PM) shadeslayer: at the very end
(12:42:09 PM) shadeslayer: or you can do as tech2077 did.. install python-software-properties
(12:42:37 PM) shadeslayer: done?
(12:42:52 PM) shadeslayer: the basic thing here is to add the kubuntu beta ppa
(12:43:10 PM) shadeslayer: well.. after that just : apt-get update
(12:43:27 PM) shadeslayer: then log out using ctrl+D
(12:43:38 PM) shadeslayer: then pdebuild again
(12:43:54 PM) shadeslayer: and this time , it should work :D
(12:45:08 PM) shadeslayer: Quintasan: ready to fire away?
(12:45:25 PM) Quintasan: hmm I think yes
(12:45:27 PM) ClassBot: penguin42 asked: So once you've built all of this stuff as Ninjas do you have a set of tests?
(12:45:37 PM) shadeslayer: penguin42: yes!
(12:45:51 PM) shadeslayer: penguin42: lintian takes care of most of the errors
(12:46:10 PM) shadeslayer: it checks the packaging for defects that the ninjas might have ignored
(12:47:07 PM) Quintasan: So, fist of all, Project Neon was/is a very Kool thing that provides users with nightly (unstable) builds of KDE SC. This new users that are up for testing can work with bleeding edge changes in KDE without compiling the whole source by themselves.
(12:47:23 PM) shadeslayer: ok after the deps are downloaded and unpacked,pbuilder builds the package and you get .debs in you pbuilder result dir
(12:47:31 PM) shadeslayer: which is basically all about ninja packaging :D
(12:47:56 PM) shadeslayer: just as a example,kdelibs takes 2-3 hours to build
(12:47:57 PM) Quintasan: shadeslayer: you forgot about uploading to hyper secret ppa
(12:48:07 PM) shadeslayer: ^^
(12:48:11 PM) ext2 is now known as ext3
(12:48:14 PM) shadeslayer: ah yes.. the hyper secret ppa
(12:48:48 PM) shadeslayer: if you want to get your name spoken in the elite circles of ninjas,grab one of the ninjas and ask him to review your work
(12:49:04 PM) shadeslayer: after a review,you get access to super secret ppa
(12:49:13 PM) shadeslayer: where you can test your builds in the ppa!
(12:49:30 PM) shadeslayer: s/hyper/ultimate
(12:49:35 PM) shadeslayer: as apachelogger pointed out
(12:50:54 PM) Quintasan: okay
(12:51:09 PM) shadeslayer: quickly neon :D
(12:51:53 PM) Quintasan: as I worte earlier, Project Neon will provide you with latest builds of KDE SC. Some time ago apachelogger wrote some magic code in Ruby but it won't work now and we have decided to port it to Launchpad Recipes.
(12:52:10 PM) shadeslayer: Which is in a beta stage as well
(12:53:55 PM) Quintasan: This is where packaging comes in handy. I do belive that latest builds should be provided with ever possible feature.
(12:54:52 PM) Quintasan: That's why we (me, shadeslayer, apachelogger) need to check for every additional dependecies, add them, rewrite rules and install files.
(12:57:12 PM) Quintasan: Well, without boring stuff. Come to #project-neon and help us with out uber 1337 mission of providing the best nightly builds of KDE SC
(12:58:10 PM) Quintasan: simulacrum: Well, there is a page in Kubuntu Wiki but it is currently empty, I have been thinking about specs for the past few days
(12:58:59 PM) Quintasan: simulacrum: feel free to ask us anything in #project-neon, we currently need hands to sort out additional dependencies for our packages.
(12:59:43 PM) Quintasan: Thank shadeslayer for teaching you those all things, you can help us right away
(12:59:46 PM) Quintasan: :)