Ubuntu Wiki

   1 [07:01] <asac> good morning!
   2 [07:02] <gotunandan> good morning !
   3 [07:02] <asac> anyone here ;)?
   4 [07:02] <brand0con> good evening(?)
   5 [07:02] <asac> ah good.
   6 [07:02] <gotunandan> almost good afternoon here (india)
   7 [07:03] <amartani> good morning!
   8 [07:03] <Scienceman123> Hello
   9 [07:04] <Scienceman123> 2 am here
  10 [07:04] <asac> i almost wondered if anyone would show up at this time ;) ... but great
  11 [07:05] <asac> ok lets get started. please be a bit patient with me ... its really early and usually i wouldnt be awake now ;)
  12 [07:05] <Scienceman123> Me neither :P
  13 [07:05] <asac> i really appreciate that you came then ;)
  14 [07:06] <asac> good thing is that - afaik - there is no session directly after this session, so in case it takes a bit longer we won't thrown out. (though i will try to keep it within one hour)
  15 [07:07] <asac> so basiclly todays packaging training session is supposed to be about "Mozilla packaging techniques (extensions, patchsystems, bzr)"
  16 === MT- is now known as MTeck
  17 [07:07] <asac> why mozilla?
  18 [07:08] <asac> Mozillateam takes care of a variety of packages surrounding mozilla software
  19 [07:09] <asac> + we have pretty big beasts like tbird/sunbird/firefox/xulrunner
  20 [07:09] <asac> + our expertise touches various packages that interface with mozilla in one way or another ... (cairo, fontconfig,) (plugins and embedding apps)
  21 [07:10] <asac> + xulrunner applications: fennec, prism
  22 [07:10] <asac> (and probably know, but I can't reember)
  23 [07:10] <asac> more;)
  24 [07:10] <asac> + lots of extensions
  25 [07:10] <asac> + helper tools to standardize and easy mozilla package maintenance
  26 [07:10] <asac> s/easy/ease/
  27 [07:12] <asac> so thats quite a lot work; i want to show you know what techniques were established to make the package maintenance as efficient as possible
  28 [07:12] <asac> one more adverstiment thing: we have our channel in #ubuntu-mozillteam ... so any questions wrt mozilla packages are best anwered there
  29 [07:13] <asac> so today i planned to how we achieve a few of the various objectives; in particular:
  30 [07:14] <asac> efficiency and transparency for upstream
  31 [07:14] <asac> (damn my typo is still bad ... if you get confused by too many typos let me know ;))
  32 [07:15] <asac> wrt to efficiency I want to look at various aspects:
  33 [07:16] <asac> 1. how to improve efficiency of a team
  34 [07:16] <asac> 2. how to maintain patches in an efficient way - show tricks
  35 [07:17] <asac> 3. how source tree layout can contribute to efficiency
  36 [07:17] <asac> 4. how we make backporting efficient
  37 === MTeck is now known as MT-
  38 [07:18] <asac> 5. and special requirements we get by providing daily builds
  39 [07:18] <asac> so i want to start with section one of todays session ;) ... "Team & Tools"
  40 [07:19] <asac> efficient team maintenance depends on a shared set of tools and best practices
  41 [07:19] <asac> basically the goal is:
  42 [07:20] <asac> anyone in a Team should be able to fix/adjust/extend packaging branches without thinking twice
  43 [07:20] <asac> this
  44 [07:20] <asac> helps to be better leverage learning effects, but also reduces discussion about such  meta topics itself to a minimum
  45 [07:21] <asac> Examples (team & tools):
  46 [07:21] <asac> + mozillateam uses bzr - discussing the benefits of a distributed VCS for a team is a bit beyond the scope of this session, but probably feel obvious to most, right?
  47 [07:22] <asac> + something less obvious, which helps to improve team efficiency is the use of the same changelog format
  48 [07:23] <asac> doesnt sound like a big deal, but besides from improving readability it also helps
  49 [07:24] <asac> to avoid regular discussions about how this or that format is better ... so team members can focus on the real work ;)
  50 [07:25] <asac> two examples of changelog formats used; first the comon one:
  51 [07:25] <asac>  * path/to/file: what did you do
  52 [07:25] <asac> that one is pretty common, but for mozilla (at least the big beast apps mentioned above) it turned out to be suboptimal
  53 [07:26] <asac> basically because most of the work we do is related to patches: updating patches, adding patches, removing patches
  54 === noodles775 is now known as noodles
  55 === noodles is now known as noodles775
  56 [07:26] <asac> also we explicitly add some meta information to our patches to increase upstream transparency
  57 [07:27] <asac> so our file names would be rather long:
  58 [07:27] <asac> * debian/patches/bz386904_config_rules_install_dist_files.patch ...
  59 [07:27] <asac> if you have a bunch of those files touched the changelog becomes unreadable ... thats why mozillateam took a different approach and first state what is done and then how and what files touched
  60 [07:28] <asac> * i did this
  61 [07:28] <asac>   - update debian/patches/bz386904_config_rules_install_dist_files.patch
  62 [07:28] <asac>   - update debian/patches/series
  63 [07:28] <asac>   - drop debian/patches/yetanother_patch
  64 [07:29] <asac> you can see this format being used in most mozilla related packages in ubuntu i would think
  65 [07:29] <asac> next thing is similar trivial, but even more important: "Branch names"
  66 [07:29] <asac> agreeing on how to name branches improves efficiency quite a lot
  67 [07:30] <asac> ideally, anyone in the team can find any branches by typing in the location bar or console directly
  68 [07:30] <asac> our format is:
  69 [07:30] <asac>  Release branches: lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head]
  70 [07:30] <asac> (release branches are the main branches that are used to release uploads from)
  71 [07:31] <asac>  Topic branches lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head].${TOPIC_NAME}
  72 [07:31] <asac> so for example: lp:~mozillateam/firefox/firefox-3.0.head
  73 [07:31] <asac> TEAM = mozillateam
  74 [07:31] <asac> PROJECT = firefox
  75 [07:31] <asac> PACKAGE = firefox-3.0 (source package)
  76 [07:32] <asac> RELEASE = hardy|intrepid|jaunty ... "head"
  77 [07:32] <asac> head is a special release name somehow which we use to signal that its the branch for the current development release
  78 [07:33] <asac> soinstead of using .karmic now we are using .head; when we come close to a release we cut off stable releaes branches where we dont do much development on anymore
  79 [07:33] <asac> so if you have a "real" release it means in mozillateam land that its a stable maintenance branch
  80 [07:34] <asac> two more things before we do a quick exercise for the team & tools section
  81 [07:34] <asac> we also do "release commits":
  82 [07:34] <asac> -> this boosts efficiency because you can easily spot revisions that reflect any given package version
  83 [07:35] <asac> we use bzr-builddeb accors the board as a conventient way to produce sources/binaries etc.
  84 [07:36] <asac> and we use debian/rules get-orig-source quite heavily to produce tarballs directly from upstream repositories (e.g. snapshots/dailies/releases)
  85 [07:36] <asac> hmm... time is running fast. any questions for now?
  86 [07:37] <asac> maybe lets do a simple exercise about "branch names":
  87 [07:37] <asac> guess the branch url for the nss jaunty release branch and bzr branch it
  88 [07:37] <asac> anyone can guess the name ;)?
  89 [07:38] <asac> so the template is: lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head]
  90 [07:38] <asac> bzr branch lp:~mozillateam/nss/nss.jaunty ;)
  91 [07:40] <asac> everyone got that branch ;O)?
  92 [07:41] <asac> ok ... so for next excersize you similarly should branch the "nspr" karmic branch as we want to prepare a release ;)
  93 [07:41] <asac> bzr branch lp:~mozillateam/nspr/nspr.head
  94 [07:43] <asac> so what you usually do during development is to keep the changelog at UNRELEASED (check out debian/changelog)
  95 [07:43] <asac> currently its karmic, but assume its all the time UNRELEASED until the real upload
  96 [07:43] <asac> so when you are done with all the branchwork you do a release commits. Thats basically two commands:
  97 [07:44] <asac> 1. dch -r -DRELEASE (e.g. dch -r -Dkarmic)
  98 [07:44] <asac> -> this updates the changelog timestamps and flipps UNRELEASED to karmic (or whatever you specified in RELEASE
  99 [07:44] <asac> 2. debcommit -r
 100 [07:45] <asac> -> this does the actual release commit. it autoamtically creates a bzr commit log reading like: "releasing 0.0-0ubuntu1"
 101 [07:45] <asac> this helps anyone in the team later to spot the actual release by just looking at changelog
 102 [07:46] <asac> also it creates a tag using the version in changelog - which is a good thing, but the mozillateam usually does not use tags for that
 103 [07:46] <asac> so when you did those two commands you basically have prepared a nspr release in mozilla-team-style
 104 [07:47] <asac> so seince you are not a mozillateam member, where would you push the branch to in order to contribute?
 105 [07:47] <asac> answer is simple: push your branch up to launchpad using an appropriate topic branch name
 106 [07:47] <asac> and request a merge
 107 [07:48] <asac> Topic branches lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head].${TOPIC_NAME}
 108 [07:48] <asac> so for this it could be: lp:~yourname/nss/nss.head.4.8-0ubuntu1
 109 [07:48] <asac> ok thats it for teams and tools :)
 110 [07:49] <asac> since time is already running low i will a bunch of things i said to say ...
 111 [07:49] <asac> lets go from team & tools to patches
 112 [07:49] <asac> in particular its important that adding patches also causes long term maintenance pain
 113 [07:50] <asac> especially in an environmnet where you want to provide daily builds based on a really active upstream
 114 [07:50] <asac> like firefox
 115 [07:50] <asac> (example: https://edge.launchpad.net/~ubuntu-mozilla-daily/+archive)
 116 [07:51] <asac> one thing to wake you up:
 117 [07:51] <asac> + seemingly easy patches can become a pain; see http://bazaar.launchpad.net/~chromium-team/chromium-v8/chromium-v8.head/annotate/37/debian/patches/enable_soname.patch
 118 [07:51] <asac> thats a patch i recently added to our chromium v8 branch (the chrome javascript engine)
 119 [07:51] <asac> anyone has any idea whats so painful about that patch?
 120 [07:53] <asac> the problem is not that its a big patch that is hard to figure how to port in caes upstream changes the underlying file
 121 [07:53] <asac> the problem here is that it touches a context that frequently changes: e.g. MINOR_VERSION, BUILD_NUMBER, PATCH_LEVEL
 122 [07:54] <asac> those regularly change and i am sure if that patch would have been kept we would get build failures every few days in our ~chromium-daily ppa
 123 [07:54] <asac> anyone can think of a way to prevent that?
 124 [07:55] <asac> here is the better approach (even if its a bit ugly its much less painful):
 125 [07:55] <asac> http://bazaar.launchpad.net/~chromium-team/chromium-v8/chromium-v8.head/revision/39#debian/rules
 126 [07:55] <asac> so basically we dont put a patch in there, but rather do the change using sed from debian/rules :)
 127 [07:56] <asac> ok another thing about patches ...
 128 [07:56] <asac> + dpatch-edit-patch and simple-patchsys is suboptimal approach for a) large packages and b) medium packages that are work intensive
 129 [07:56] <asac>  - example: adjusting one patch in xulrunner 1.8 (even smaller than 1.9) with dpatch-edit-patch run took several minutes
 130 [07:56] <asac>  thats time better spend on other things i am sure
 131 [07:57] <asac> mozillateam uses quilt ... its a bit more to learn up front, but you can work within a source tree and dont need to regularly create full copies of it and so on
 132 [07:58] <asac> as an exercise you could try to adjust the soname patch from above using quilt (not now though).
 133 [07:58] <asac> i will stay in the channel if you have questions. or join #ubuntu-mozillateam
 134 [07:59] <asac> i will skip two topics now because of time constraints and move directly to last topic: tricks on how to make backports easy
 135 [08:00] <asac> about backporting
 136 [08:00] <asac> especially for the dailies we dont want to maintain branches for each and every ubuntu releaes
 137 [08:00] <asac> instead we take care that .head branches just work everywhere
 138 [08:00] <asac> for that we use various tricks ;)
 139 [08:01] <asac> one trick is to enable/disable features based on the version that is available
 140 [08:01] <asac> so if firefox 3.0 needs PACKAGE b >= VERSION 1 for a certain feature
 141 [08:01] <asac> we wouldnt add a hard depent to debian/control like b >= VERSION1
 142 [08:01] <asac> instead we just add an unversoined dependency on package b to debian/control
 143 [08:02] <asac> and then in debian/rules check the installed version and enable/disable a certain feature on demand
 144 [08:02] <asac> you can check that yourself in rules and control of http://code.launchpad.net/~mozillateam/xulrunner/xulrunner-1.9.2.head
 145 [08:02] <asac> look for nspr/nss and cairo and sqlite
 146 [08:04] <asac> ok since time is off, go ahead and ask me
 147 [08:04] <asac> include my nick so i can see your questions
 148 [08:04] <asac> the notes i prepared for this session can be found here: http://paste.ubuntu.com/236213/
 149 [08:04] <asac> maybe it helps you to re-read something
 150 [08:05] <asac> if you do the exercises and need help, let me know
 151 [08:05] <asac> if not today here, you can ask i #ubuntu-mozillateam
 152 [08:05] <asac> i didnt touch extension maintainence this cycle because i did it like 4 last time i did a session
 153 [08:05] <asac> if you are interested in helping out doing extensions, jump in our channel
 154 [08:05] <asac> we have lots of work todo this cycle
 155 [08:06] <asac> and extensions are an easy starter
 156 [08:06] <asac> ok thanks
 157 [08:06] <asac> !