Ubuntu Wiki

   1 [13:00] <funkyHat> Hi:)
   2 [13:01] <@asac> hi all
   3 [13:03] <@asac> so i had offered to run a packaging training session today
   4 [13:03]  * ianto applauds
   5 [13:03]  * funkyHat bounces
   6 [13:03] <@asac> in particular about patchsystems
   7 [13:03]  * juanje applauds
   8 [13:03] <@asac> so who is here ;)?
   9 [13:04]  * ianto looks around
  10 [13:04] <ianto> me
  11 [13:04] <juanje> me
  12 [13:04] <kmdm> here :)
  13 [13:04] <funkyHat> asac: will you be joining #ubuntu-classroom-chat, or should we just ask questions in here?
  14 [13:04]  * jp_sf raising hand
  15 [13:04] <@asac> i would like to have this kind of interactive so please ask here
  16 [13:05] <@asac> so anyone worked on packages has probably noticed that the sources usually ship a upstream tarball and a diff.gz
  17 [13:06] <Ludoza> evening asac
  18 [13:06] <@asac> so while this works great for packages where you dont need to modify the upstream code base, it can be tricky if you need to maintain multiple patches
  19 [13:07] <@asac> patch systems are there to help you keep those changes in a separate and clean fashion
  20 [13:08] <@asac> so if you start to work on a package that has a patchsystem it usually has a debian/patches/ directory
  21 [13:08] <@asac> maybe check out the source for gwibber: apt-get source gwibber
  22 [13:09] <oldman_> :)
  23 [13:09] <@asac> it uses a so called "simple" patchsystem
  24 [13:10] <@asac> simple patchsystem is simple in the sense that you just drop your patches in that directory
  25 [13:10] <apw> asac w
  26 [13:10] <funkyHat> I don't see debian/patches :/
  27 [13:10] <@asac> and the order used to apply the patches is solely determined by the lexicographical order
  28 [13:10] <apw> which version of gwibber?  i see no patches in jaunty
  29 [13:10] <@asac> 0.8
  30 [13:10] <ianto> funkyHat: cd gwibber-0.8/debian/patches
  31 [13:11] <@asac> apw: after i did apt-get source gwibber, there is a gwibber-0.8/debian/patches directory
  32 [13:11] <funkyHat> Ah, I'm looking at 0.9.2
  33 [13:11] <oldman_> yeah gwibber-0.9.2 on karmic also
  34 [13:11] <apw> yeah in 0.9.2 from jaunty its gone, and looks to be in bzr
  35 [13:11] <ianto> 3rd party repo probably ;-/
  36 [13:11] <@asac> heh so you are more modern than me ;)
  37 [13:11] <mnemo> QUESTION: asac, is "simple" the same as cdbs? because if I grep for "cdbs" in gwibber sources I get some hits... otherwise, whats a good method to determine the patch system for a given package (I usually grep for the string quilt, cdbs and dpatch etc)
  38 [13:12] <funkyHat> can I apt-get source a specific version?
  39 [13:12] <@asac> ok so if you have 0.9.6 and no debian/patches, run
  40 [13:12] <@asac> dget https://edge.launchpad.net/ubuntu/jaunty/+source/gwibber/0.8-0ubuntu5/+files/gwibber_0.8-0ubuntu5.dsc
  41 [13:12] <@asac> and then dpkg-source -x gwibber*dsc
  42 [13:12] <@asac> ;)
  43 [13:12] <ianto> dget -x does the same job? ;)
  44 [13:13] <@asac> that will give you the 0.8 sources i am looking at
  45 [13:13] <@asac> ianto: could be ;)
  46 [13:13] <@asac> mnemo: yes, actually the simple patchsystem is a cdbs specific patchystem
  47 [13:13] <@asac> mnemo: you can identify packages that use the simple patchsystem by looking in debian/rules
  48 [13:14] <@asac> there is a line like: include /usr/share/cdbs/1/rules/simple-patchsys.mk
  49 [13:14] <funkyHat> Ok now I am on the same page as you :)
  50 [13:14] <persia> (and if you're very lucky, debian/README.source will also explain which patch system is in use)
  51 [13:15] <@asac> ok so everyone looks at the same package source i guess ...
  52 [13:15] <@asac> so there are now a few common you usually want to do with patches:
  53 [13:15] <@asac> 1. drop a patch
  54 [13:16] <@asac> 2. add a new patch
  55 [13:16] <@asac> 3. edit a patch
  56 [13:16] <@asac> 4. update a patch when upstream code-base changed
  57 [13:17] <@asac> so 1. is obviously simple with simple-patchsys
  58 [13:17] <@asac> just remove the patch
  59 [13:17] <@asac> sometimes - if the dropped patch interleaves with some other patch that gets later applied, this can also lead to 4.
  60 [13:18] <@asac> for adding a new patch there are two variants:
  61 [13:18] <@asac> a) just put the new patch in the debian/patches/ directory
  62 [13:19] <@asac> b) the other way is to just use the same mechanism as 3. edit a patch ... just that you use a new patchname
  63 [13:19] <@asac> so lets look at 3. edit a patch
  64 [13:20] <@asac> consider we found out that 03_locale.patch is wrong and we want to change it we would use cdbs-edit-patch to update it
  65 [13:21] <@asac> e.g. cdbs-edit-patch 03_locale_dir.patch
  66 [13:21] <@asac> when you run this, cdbs will copy the current source tree to a temp directory and apply all the patches up to the patch you want to edit
  67 [13:22] <@asac> and leave you in a shell that allow you just to edit the source tree in your preferred editor
  68 [13:23] <@asac> so for this case lets edit gwibber.desktop.in and fix some typo (change whatever you like)
  69 [13:23] <@asac> once you are happy with your change you can exit the cdbs edit shell by typing exit 0
  70 [13:24] <@asac> if you exit with a non-zero code the changes you did will not get applied
  71 [13:25] <@asac> so in case i figure that i did something wrong or want to do something else, use exit 1 or something
  72 [13:25] <@asac> any questions on how to edit patches using cdbs-edit-patch?
  73 [13:26] <funkyHat> So it's not so much editing the patch as replacing it?
  74 [13:26] <@asac> funkyHat: no its really an edit
  75 [13:27] <@asac> if you run cdbs-edit-patch PATCHNAME
  76 [13:27] <@asac> it will apply the current patch, so the previous changes are already there
  77 [13:28] <mnemo> QUESTION: asac, this "simple" patch system seems very straight forward, why would one want to complicate things further?? why use quilt or so for example where the patches have to be popped and pushed explicitly?
  78 [13:28] <funkyHat> Ok... I just tried it and the patch file is only showing my changes, nothing else
  79 [13:29] <@asac> funkyHat: if i run: 1. cdbs-edit-patch 03_locale_dir.patch 2. echo test >> gwibber.desktop.in 3. exit 0
  80 [13:29] <@asac> i see both changes in 03_locale_dir-patch
  81 [13:29] <@asac> mnemo: thats in deed a valid question. in particular if you compare it with dpatch
  82 [13:30] <@asac> mnemo: so there are a few cases where dpatch or quilt make more sense.
  83 [13:30] <@asac> dpatch allows you to do arbitrary stuff in patches ... so you are basically not constrainted patches in the sense of a "diff"
  84 [13:31] <funkyHat> asac: oh, was the original change on the _Name line?
  85 [13:31] <@asac> for instance you could create a backup of a file, then run some sed statement to replace all occurences of a string you want to change
  86 [13:31] <@asac> the tricky thing with dpatch with using scripts is that you always have to take care that the "unpatcH" operation gets you really back to the state you had before applying the patch
  87 [13:32] <mnemo> QUESTION: asac, so is there like a "holy grail" of patch systems towards which all the debian packages are migrating? (I mean is there an agreed upon "best" system and its just a matter of man power to get it standardized?)
  88 [13:34] <@asac> mnemo: no. there are valid use cases
  89 [13:34] <@asac> for other patchsystems
  90 [13:34] <@asac> let me continue with quilt
  91 [13:34] <@asac> quilt is a real patchsystem in the sense that its not really specific to ubuntu packages or cdbs
  92 [13:34] <@asac> using quilt is definitly handy if you maintain a lot of patches and in particular when you have a large source tree
  93 [13:36] <@asac> example is xulrunner-1.9 ... if we would use  simple-patchsystem there editing just a single line of a patch would create a full copy of a 350M source tree
  94 [13:36] <@asac> and then when you exit the cdbs shell it takes a couple of minutes just to update the patch
  95 [13:36] <@asac> thats usually just annoying
  96 [13:36] <@asac> so maybe lets stick to gwibber and see how we can convert that tree to use quilt ;)
  97 [13:36] <@asac> you obviously need to install the quilt package
  98 [13:38] <@asac> once you have that you could replace the simple-patchsys in debian/rules with /usr/share/cdbs/1/rules/patchsys-quilt.mk
  99 [13:38] <@asac> quilt is a bit more sophisticated and doesnt apply all patches it finds in a directory.
 100 [13:38] <@asac> instead it looks at a file called "series" in the patches directory
 101 [13:40] <@asac> so to convert gwibber we need to create a first series file manually
 102 [13:40] <@asac> which basically is just adding all files that are currently in debian/patches/
 103 [13:40] <@asac> so something like: ls debian/patches/ | grep -v series > debian/patches/series will work
 104 [13:40] <@asac> now in order to play around with quilt set the QUILT_PATCHES env variable like:
 105 [13:40] <@asac> export QUILT_PATCHES=debian/patches
 106 [13:40] <@asac> once you did that you can use a few commands ... maybe run them and see what they do:
 107 [13:41] <@asac> quilt push
 108 [13:43] <@asac>  -> this will apply the next unapplied patch in the series files
 109 [13:43] <@asac> quilt pop
 110 [13:43] <funkyHat> ooh
 111 [13:43] <@asac>  -> this will unapply the topmost applied patch
 112 [13:43] <@asac> quilt series
 113 [13:43] <@asac>  -> this will give you a list of all patches
 114 [13:43] <@asac> quilt applied
 115 [13:43] <@asac>  -> this will give you a list of already applied patches
 116 [13:43] <@asac> quilt unapplied
 117 [13:43] <@asac>  -> this will give you a list of not-yet-applied patches in the series
 118 [13:43] <@asac> so if you did what i said you now have a debian/patches/series file and running quilt push  three times will finally apply the 03_locale_dir.patch
 119 [13:43] <@asac> anyone who doesnt have that?
 120 [13:46]  * rraphink finds it useful to add "export QUILT_PATCHES=debian/patches" to his .bashrc for when there are packages using quilt 
 121 [13:46] <@asac> yes, thats a good idea if you regular work on packages
 122 [13:46] <raphink> together with the debhelper variables, too
 123 [13:46] <@asac> personally, i sometimes use quilt for upstream trees too ... so its not always the same
 124 [13:46] <@asac> ok so lets look how to edit a patch ;)
 125 [13:46] <bac> asac: is there a shortcut to apply all patches at once so you don't have to push n times?
 126 [13:46] <@asac> to edit a patch you basically use quilt push/pop until the patch you want to edit is on top
 127 [13:46] <@asac> (quilt top)
 128 [13:46] <@asac> bac: to apply all patches at once you can use quilt push -a
 129 [13:46] <@asac> similarly you can use quilt pop -a
 130 [13:46] <@asac> or you can even name a patch
 131 [13:46] <bac> thx
 132 [13:48] <@asac> e.g. quilt push 02_twitter_api_fix.patch
 133 [13:48] <@asac> so lets assume we want to edit the 03_locale_dir.patch
 134 [13:48] <@asac> quilt push 03_locale_dir.patch
 135 [13:48] <@asac> ...
 136 [13:48] <@asac> Now at patch 03_locale_dir.patch
 137 [13:48] <@asac> when you get that you can look what the current diff of that patch is by running:
 138 [13:48] <@asac> quilt diff
 139 [13:51] <@asac> now the important part. quilt is great becaues it doesnt need to diff a full tree (so its fast)
 140 [13:51] <@asac> once negative side-affect is that you explicitly have to tell quilt to include a certain file in the patch _before_ you edit it
 141 [13:51] <@asac> so in we want to edit gwibber.desktop.in
 142 [13:51] <@asac> (like we did for simple patch)
 143 [13:51] <@asac> you have to run quilt add gwibber.desktop.in _before_ you edit the patch
 144 [13:51] <@asac> err edit the file ;)
 145 [13:51] <@asac> thats really important and you easily get trapped if you dont run quilt add path/to/filename first
 146 [13:51] <@asac> because if you forget it its not really easy for you to get the changes backed out  and so on
 147 [13:51] <@asac> so after running quilt add gwibber.desktop.in
 148 [13:51] <@asac> you can again edit that file, like: echo append >> gwibber.desktop.in
 149 [13:51] <@asac> after editing it you should see your changes in quilt diff
 150 [13:53] <@asac> note that quilt diff shows the full diff of the current topmost patch together with your new changes
 151 [13:53] <@asac> so its basically a projection of how the diff would look like if you decide to refresh the patch
 152 [13:53] <@asac> so a) when you are happy with what quilt diff shows you, you need to explicitly refresh the patch like: quilt refresh
 153 [13:53] <@asac> and b) if you are unhappy with what you changed you can undo your latest changes with quilt pop -f
 154 [13:56] <@asac> any questions on how to edit patches with quilt?
 155 [13:56] <@asac> important commadns to remember (besides push/pop et al):
 156 [13:56]  * apw points asac at quilt edit, which seems to handle the adding
 157 [13:56] <@asac>  quilt add FILENAME (add a file you want to modify to to be tracked by current quilt patch)
 158 [13:56] <@asac>  quilt diff (look at how the patch looks with your current local changes)
 159 [13:56] <funkyHat> I much prefer this to simple patches :)
 160 [13:56] <@asac>  quilt refresh (update the patch to match what you see in quilt diff)
 161 [13:56] <raphink> quilt is almost like a VCS in a way
 162 [13:59] <@asac> as i said above ... if you ever used simple-patchsystem with a package that isnt tiny you will definitly look for something else
 163 [13:59] <@asac> and quilt is pretty much the only option you have ... unless you dont want to use a patchsystem at all ;)
 164 [13:59] <@asac> that said ... simple patchsystem will work for most packages in the archive
 165 [13:59] <raphink> unless you're not using cdbs ;)
 166 [13:59] <@asac> however, if you are also doing upstream development and want something lightweight to maintain patches while tracking upstream VCS as a baseline quilt is also really helpful
 167 [13:59] <raphink> i.e. converting a package to simple-patchsys often means converting it first to cdbs
 168 [14:00] <Rail> asac: is there any easy (automatic) way to convert feature branches into cdbs/quilt patches?
 169 [14:01] <Rail> for git-builddeb for example
 170 [14:01] <@asac> Rail: that depends on how you want to export feature branches. so exporting commits is quite easy in git
 171 [14:01] <@asac> e.g. lets say git tree has a tag release1.0 ... and you committed a few things on top that you want as individual patches you could use git format-patch release1.0
 172 [14:01] <@asac> then you can just copy the patches in debian/patches/
 173 [14:01] <@asac> and if you use quilt also add that to series
 174 [14:01] <Rail> yes, it's easy to git diff but sometimes you can forget to di this
 175 [14:02] <Rail> *do
 176 [14:02] <@asac> Rail: git format-patch
 177 [14:02] <@asac> exports commits
 178 [14:02] <@asac> if you really have feature branches you need to bzr diff them anyway i guess
 179 [14:03] <@asac> i dont know about anything automated for that ... but maybe i am wrong
 180 [14:03] <Rail> thanks
 181 [14:05] <Rail> and one advise from me, don't forget to use patch tags ;)
 182 [14:05] <Rail> https://wiki.ubuntu.com/UbuntuDevelopment/PatchTaggingGuidelines
 183 [14:05] <@asac> yeah. patch tagging is nice
 184 [14:05] <@asac> in anycase, please also properly document stuff in changelog
 185 [14:05] <@asac> personally i think patch tagging is good, but as long as that info is easily discoverable thats good as well
 186 [14:05] <@asac> changelog entry can help
 187 [14:06] <@asac> also the filename can indicate launchpad bug id or upstream bug tracker
 188 [14:06] <@asac> but yeah, if you do new patches, please use https://wiki.ubuntu.com/UbuntuDevelopment/PatchTaggingGuidelines
 189 [14:08] <@asac> oh ... i think i forgot to say how to add new patches with quilt/simple patch
 190 [14:08] <@asac> so since time is over just quick:
 191 [14:08] <@asac> the procedure is the same as for editing patches, just that for simple patchsystem you us a name that isnt used yet
 192 [14:08] <@asac> and for quilt you first run quilt new NEW-PATCH-NAME.patch
 193 [14:08] <mnemo> QUESTION: when apps are built and packaged using bazaar LP branches, do they still use a patch system?
 194 [14:08] <@asac> quilt will add a new patch after the current top patch ... so if you want to add a new patch that is applied before everything else is applied you can first pop all patches and then use quilt new
 195 [14:08] <@asac> mnemo: yes. patchsystems are not really replacable by VCS ....
 196 [14:08] <@asac> the only way that would be possible would be if you had a feature branch for each patch
 197 [14:09] <@asac> however, this is quite problematic because once the patches interleave and touch same files
 198 [14:09] <@asac> you need an order of feature branches and so on
 199 [14:09] <@asac> for bzr there is the loom extension
 200 [14:09] <@asac> which somehow tries to do that, but personally i found it rather high-overhead
 201 [14:09] <@asac> so my personal preference is to just maintain a bzr branch with just the debian/ directory
 202 [14:09] <@asac> and still maintain patches therein
 203 [14:10]  * Rail prefers the same method :)
 204 [14:13] <@asac> of course how you produce those patches is left open to the author ... if oyu have multiple long running feature development going on you might want to use feature branches and regular export those
 205 [14:13] <@asac> but saying that those are th eofficial sources for producing a package would make it harder for others to help out
 206 [14:13] <@asac> hope that answers the question
 207 [14:13] <mnemo> yup
 208 [14:13] <@asac> anything else?
 209 [14:13] <@asac> 3
 210 [14:13] <@asac> 2
 211 [14:13] <@asac> 1
 212 [14:13] <@asac> done... thanks all
 213 [14:13] <cking> asac, thanks!
 214 [14:13] <mnemo> thanks a lot asac
 215 [14:13] <micmord> asac: thanks
 216 [14:13] <Rail> thanks a lot asac
 217 [14:13] <oldman_> great job asac
 218 [14:13] <dholbach> thanks asac!
 219 [14:13] <@asac> you can find me in #ubuntu-mozillateam or #ubuntu-desktop if more questions come up later or tomorrow
 220 [14:13] <hifi> thanks, I'll read the logs
 221 [14:13] <@asac> dont hesitate to ask
 222 [14:16] <@asac> thanks