Next Session: TBD |
1 [07:01] <asac> good morning!
2 [07:02] <gotunandan> good morning !
3 [07:02] <asac> anyone here ;)?
4 [07:02] <brand0con> good evening(?)
5 [07:02] <asac> ah good.
6 [07:02] <gotunandan> almost good afternoon here (india)
7 [07:03] <amartani> good morning!
8 [07:03] <Scienceman123> Hello
9 [07:04] <Scienceman123> 2 am here
10 [07:04] <asac> i almost wondered if anyone would show up at this time ;) ... but great
11 [07:05] <asac> ok lets get started. please be a bit patient with me ... its really early and usually i wouldnt be awake now ;)
12 [07:05] <Scienceman123> Me neither :P
13 [07:05] <asac> i really appreciate that you came then ;)
14 [07:06] <asac> good thing is that - afaik - there is no session directly after this session, so in case it takes a bit longer we won't thrown out. (though i will try to keep it within one hour)
15 [07:07] <asac> so basiclly todays packaging training session is supposed to be about "Mozilla packaging techniques (extensions, patchsystems, bzr)"
16 === MT- is now known as MTeck
17 [07:07] <asac> why mozilla?
18 [07:08] <asac> Mozillateam takes care of a variety of packages surrounding mozilla software
19 [07:09] <asac> + we have pretty big beasts like tbird/sunbird/firefox/xulrunner
20 [07:09] <asac> + our expertise touches various packages that interface with mozilla in one way or another ... (cairo, fontconfig,) (plugins and embedding apps)
21 [07:10] <asac> + xulrunner applications: fennec, prism
22 [07:10] <asac> (and probably know, but I can't reember)
23 [07:10] <asac> more;)
24 [07:10] <asac> + lots of extensions
25 [07:10] <asac> + helper tools to standardize and easy mozilla package maintenance
26 [07:10] <asac> s/easy/ease/
27 [07:12] <asac> so thats quite a lot work; i want to show you know what techniques were established to make the package maintenance as efficient as possible
28 [07:12] <asac> one more adverstiment thing: we have our channel in #ubuntu-mozillteam ... so any questions wrt mozilla packages are best anwered there
29 [07:13] <asac> so today i planned to how we achieve a few of the various objectives; in particular:
30 [07:14] <asac> efficiency and transparency for upstream
31 [07:14] <asac> (damn my typo is still bad ... if you get confused by too many typos let me know ;))
32 [07:15] <asac> wrt to efficiency I want to look at various aspects:
33 [07:16] <asac> 1. how to improve efficiency of a team
34 [07:16] <asac> 2. how to maintain patches in an efficient way - show tricks
35 [07:17] <asac> 3. how source tree layout can contribute to efficiency
36 [07:17] <asac> 4. how we make backporting efficient
37 === MTeck is now known as MT-
38 [07:18] <asac> 5. and special requirements we get by providing daily builds
39 [07:18] <asac> so i want to start with section one of todays session ;) ... "Team & Tools"
40 [07:19] <asac> efficient team maintenance depends on a shared set of tools and best practices
41 [07:19] <asac> basically the goal is:
42 [07:20] <asac> anyone in a Team should be able to fix/adjust/extend packaging branches without thinking twice
43 [07:20] <asac> this
44 [07:20] <asac> helps to be better leverage learning effects, but also reduces discussion about such meta topics itself to a minimum
45 [07:21] <asac> Examples (team & tools):
46 [07:21] <asac> + mozillateam uses bzr - discussing the benefits of a distributed VCS for a team is a bit beyond the scope of this session, but probably feel obvious to most, right?
47 [07:22] <asac> + something less obvious, which helps to improve team efficiency is the use of the same changelog format
48 [07:23] <asac> doesnt sound like a big deal, but besides from improving readability it also helps
49 [07:24] <asac> to avoid regular discussions about how this or that format is better ... so team members can focus on the real work ;)
50 [07:25] <asac> two examples of changelog formats used; first the comon one:
51 [07:25] <asac> * path/to/file: what did you do
52 [07:25] <asac> that one is pretty common, but for mozilla (at least the big beast apps mentioned above) it turned out to be suboptimal
53 [07:26] <asac> basically because most of the work we do is related to patches: updating patches, adding patches, removing patches
54 === noodles775 is now known as noodles
55 === noodles is now known as noodles775
56 [07:26] <asac> also we explicitly add some meta information to our patches to increase upstream transparency
57 [07:27] <asac> so our file names would be rather long:
58 [07:27] <asac> * debian/patches/bz386904_config_rules_install_dist_files.patch ...
59 [07:27] <asac> if you have a bunch of those files touched the changelog becomes unreadable ... thats why mozillateam took a different approach and first state what is done and then how and what files touched
60 [07:28] <asac> * i did this
61 [07:28] <asac> - update debian/patches/bz386904_config_rules_install_dist_files.patch
62 [07:28] <asac> - update debian/patches/series
63 [07:28] <asac> - drop debian/patches/yetanother_patch
64 [07:29] <asac> you can see this format being used in most mozilla related packages in ubuntu i would think
65 [07:29] <asac> next thing is similar trivial, but even more important: "Branch names"
66 [07:29] <asac> agreeing on how to name branches improves efficiency quite a lot
67 [07:30] <asac> ideally, anyone in the team can find any branches by typing in the location bar or console directly
68 [07:30] <asac> our format is:
69 [07:30] <asac> Release branches: lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head]
70 [07:30] <asac> (release branches are the main branches that are used to release uploads from)
71 [07:31] <asac> Topic branches lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head].${TOPIC_NAME}
72 [07:31] <asac> so for example: lp:~mozillateam/firefox/firefox-3.0.head
73 [07:31] <asac> TEAM = mozillateam
74 [07:31] <asac> PROJECT = firefox
75 [07:31] <asac> PACKAGE = firefox-3.0 (source package)
76 [07:32] <asac> RELEASE = hardy|intrepid|jaunty ... "head"
77 [07:32] <asac> head is a special release name somehow which we use to signal that its the branch for the current development release
78 [07:33] <asac> soinstead of using .karmic now we are using .head; when we come close to a release we cut off stable releaes branches where we dont do much development on anymore
79 [07:33] <asac> so if you have a "real" release it means in mozillateam land that its a stable maintenance branch
80 [07:34] <asac> two more things before we do a quick exercise for the team & tools section
81 [07:34] <asac> we also do "release commits":
82 [07:34] <asac> -> this boosts efficiency because you can easily spot revisions that reflect any given package version
83 [07:35] <asac> we use bzr-builddeb accors the board as a conventient way to produce sources/binaries etc.
84 [07:36] <asac> and we use debian/rules get-orig-source quite heavily to produce tarballs directly from upstream repositories (e.g. snapshots/dailies/releases)
85 [07:36] <asac> hmm... time is running fast. any questions for now?
86 [07:37] <asac> maybe lets do a simple exercise about "branch names":
87 [07:37] <asac> guess the branch url for the nss jaunty release branch and bzr branch it
88 [07:37] <asac> anyone can guess the name ;)?
89 [07:38] <asac> so the template is: lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head]
90 [07:38] <asac> bzr branch lp:~mozillateam/nss/nss.jaunty ;)
91 [07:40] <asac> everyone got that branch ;O)?
92 [07:41] <asac> ok ... so for next excersize you similarly should branch the "nspr" karmic branch as we want to prepare a release ;)
93 [07:41] <asac> bzr branch lp:~mozillateam/nspr/nspr.head
94 [07:43] <asac> so what you usually do during development is to keep the changelog at UNRELEASED (check out debian/changelog)
95 [07:43] <asac> currently its karmic, but assume its all the time UNRELEASED until the real upload
96 [07:43] <asac> so when you are done with all the branchwork you do a release commits. Thats basically two commands:
97 [07:44] <asac> 1. dch -r -DRELEASE (e.g. dch -r -Dkarmic)
98 [07:44] <asac> -> this updates the changelog timestamps and flipps UNRELEASED to karmic (or whatever you specified in RELEASE
99 [07:44] <asac> 2. debcommit -r
100 [07:45] <asac> -> this does the actual release commit. it autoamtically creates a bzr commit log reading like: "releasing 0.0-0ubuntu1"
101 [07:45] <asac> this helps anyone in the team later to spot the actual release by just looking at changelog
102 [07:46] <asac> also it creates a tag using the version in changelog - which is a good thing, but the mozillateam usually does not use tags for that
103 [07:46] <asac> so when you did those two commands you basically have prepared a nspr release in mozilla-team-style
104 [07:47] <asac> so seince you are not a mozillateam member, where would you push the branch to in order to contribute?
105 [07:47] <asac> answer is simple: push your branch up to launchpad using an appropriate topic branch name
106 [07:47] <asac> and request a merge
107 [07:48] <asac> Topic branches lp:~${TEAM}/${PROJECT}/${PACKAGE}.[${RELEASE}|head].${TOPIC_NAME}
108 [07:48] <asac> so for this it could be: lp:~yourname/nss/nss.head.4.8-0ubuntu1
109 [07:48] <asac> ok thats it for teams and tools :)
110 [07:49] <asac> since time is already running low i will a bunch of things i said to say ...
111 [07:49] <asac> lets go from team & tools to patches
112 [07:49] <asac> in particular its important that adding patches also causes long term maintenance pain
113 [07:50] <asac> especially in an environmnet where you want to provide daily builds based on a really active upstream
114 [07:50] <asac> like firefox
115 [07:50] <asac> (example: https://edge.launchpad.net/~ubuntu-mozilla-daily/+archive)
116 [07:51] <asac> one thing to wake you up:
117 [07:51] <asac> + seemingly easy patches can become a pain; see http://bazaar.launchpad.net/~chromium-team/chromium-v8/chromium-v8.head/annotate/37/debian/patches/enable_soname.patch
118 [07:51] <asac> thats a patch i recently added to our chromium v8 branch (the chrome javascript engine)
119 [07:51] <asac> anyone has any idea whats so painful about that patch?
120 [07:53] <asac> the problem is not that its a big patch that is hard to figure how to port in caes upstream changes the underlying file
121 [07:53] <asac> the problem here is that it touches a context that frequently changes: e.g. MINOR_VERSION, BUILD_NUMBER, PATCH_LEVEL
122 [07:54] <asac> those regularly change and i am sure if that patch would have been kept we would get build failures every few days in our ~chromium-daily ppa
123 [07:54] <asac> anyone can think of a way to prevent that?
124 [07:55] <asac> here is the better approach (even if its a bit ugly its much less painful):
125 [07:55] <asac> http://bazaar.launchpad.net/~chromium-team/chromium-v8/chromium-v8.head/revision/39#debian/rules
126 [07:55] <asac> so basically we dont put a patch in there, but rather do the change using sed from debian/rules :)
127 [07:56] <asac> ok another thing about patches ...
128 [07:56] <asac> + dpatch-edit-patch and simple-patchsys is suboptimal approach for a) large packages and b) medium packages that are work intensive
129 [07:56] <asac> - example: adjusting one patch in xulrunner 1.8 (even smaller than 1.9) with dpatch-edit-patch run took several minutes
130 [07:56] <asac> thats time better spend on other things i am sure
131 [07:57] <asac> mozillateam uses quilt ... its a bit more to learn up front, but you can work within a source tree and dont need to regularly create full copies of it and so on
132 [07:58] <asac> as an exercise you could try to adjust the soname patch from above using quilt (not now though).
133 [07:58] <asac> i will stay in the channel if you have questions. or join #ubuntu-mozillateam
134 [07:59] <asac> i will skip two topics now because of time constraints and move directly to last topic: tricks on how to make backports easy
135 [08:00] <asac> about backporting
136 [08:00] <asac> especially for the dailies we dont want to maintain branches for each and every ubuntu releaes
137 [08:00] <asac> instead we take care that .head branches just work everywhere
138 [08:00] <asac> for that we use various tricks ;)
139 [08:01] <asac> one trick is to enable/disable features based on the version that is available
140 [08:01] <asac> so if firefox 3.0 needs PACKAGE b >= VERSION 1 for a certain feature
141 [08:01] <asac> we wouldnt add a hard depent to debian/control like b >= VERSION1
142 [08:01] <asac> instead we just add an unversoined dependency on package b to debian/control
143 [08:02] <asac> and then in debian/rules check the installed version and enable/disable a certain feature on demand
144 [08:02] <asac> you can check that yourself in rules and control of http://code.launchpad.net/~mozillateam/xulrunner/xulrunner-1.9.2.head
145 [08:02] <asac> look for nspr/nss and cairo and sqlite
146 [08:04] <asac> ok since time is off, go ahead and ask me
147 [08:04] <asac> include my nick so i can see your questions
148 [08:04] <asac> the notes i prepared for this session can be found here: http://paste.ubuntu.com/236213/
149 [08:04] <asac> maybe it helps you to re-read something
150 [08:05] <asac> if you do the exercises and need help, let me know
151 [08:05] <asac> if not today here, you can ask i #ubuntu-mozillateam
152 [08:05] <asac> i didnt touch extension maintainence this cycle because i did it like 4 last time i did a session
153 [08:05] <asac> if you are interested in helping out doing extensions, jump in our channel
154 [08:05] <asac> we have lots of work todo this cycle
155 [08:06] <asac> and extensions are an easy starter
156 [08:06] <asac> ok thanks
157 [08:06] <asac> !