Ubuntu Wiki

   1 [16:01] <dholbach> I'm here with the amazing Jorge O. Castro, jcastro!
   2 [16:01] <jcastro> woo hoo!
   3 [16:01] <jcastro> we have been preparing real hard for this session, hope you like it!
   4 === godzilla is now known as Guest53616
   5 [16:02] <dholbach> And together we'll talk about Adopting an Upstream! Why it's fun, necessary and makes the world a better place.
   6 [16:02] <dholbach> jcastro: What's an Upstream?
   7 [16:02] <jcastro> that's a good question
   8 [16:02] <jcastro> first, when you get an ubuntu CD it comes with a bunch of great software
   9 [16:02] <jcastro> things like Firefox, Openoffice, GNOME, KDE, etc.
  10 [16:03] <jcastro> what Ubuntu does as a distribution is take all those things, assemble them into one operating system, and then ship
  11 [16:03] <jcastro> so, these projects, we call them upstream, like in a river
  12 [16:03] <jcastro> and distributions like us are downstram
  13 [16:03] <jcastro> and derived distributions like Mint would be downstream from us
  14 [16:03] <dholbach> so that's like all the software authors of those projects?
  15 [16:03] <jcastro> working with upstreams and downstreams is a critical part of ubuntu
  16 [16:03] <jcastro> right!
  17 [16:03] <dholbach> (I hope I encourage YOU ALL to ask questions too, I asked the first ones myself... :-))
  18 [16:04] <jcastro> so, for example, it's critical for us to be able to work with upstreams like Firefox to ensure that we're shipping something users will love
  19 [16:04] <jcastro> so for example this week in #ubuntu-mozillateam you'll see our Ubuntu teams working with people from Mozilla
  20 [16:04] <dholbach> So there's millions of Ubuntu users and there's thousands of Upstream projects... looks like being there for both huge groups is quite an undertaking
  21 [16:04] <jcastro> so there's a bunch of cross boundary work going on there that some people don't see
  22 [16:05] <jcastro> right, especially when you're trying to fix a problem! But we'll show you how to do that when we get there
  23 [16:05] <jcastro> you'll see that sometimes putting a bug in the right place that we can be a service to upstreams when it comes to testing.
  24 [16:06] <jcastro> and when we fix problems it's of course our duty to make sure that those fixes get back upstream so that they can improve the overall quality of their project
  25 [16:06] <jcastro> and of course, the other half is ensuring that when upstream fixes bugs that we get those out to users, because at the end of the day that's what it's all about
  26 [16:06] <jcastro> so
  27 [16:06] <jcastro> I have been working on some wiki pages to help people get started
  28 [16:07] <jcastro> https://wiki.ubuntu.com/Upstream
  29 [16:07] <jcastro> It's unreasonable to expect every upstream author to know in detail how ubuntu works
  30 [16:07] <jcastro> some are content just making their software
  31 [16:07] <dholbach> and the other way around :)
  32 [16:08] <jcastro> so I've put together a little cheat sheet here for them (you can help improve this too if you find mistakes)
  33 [16:08] <jcastro> so that they can have one place to find important things, like when we freeze, what our security policy is, etc.
  34 [16:08] <jcastro> and also give them information on how things are going on the ubuntu platform
  35 [16:08] <jcastro> so if you wanted to find out about notify osd or messaging indicators, I've listed those there
  36 [16:09] <jcastro> now
  37 [16:09] <jcastro> one thing we've all been working on is a little program to give upstreams an "in" in the distro
  38 [16:09] <jcastro> this would be a person that does know how Ubuntu works
  39 [16:09] <jcastro> and is passionate about that specific piece of software
  40 [16:10] <jcastro> and wants to work on it in Ubuntu but also be there for the upstreams to answer their questions, be an information source, and be there when things go wrong
  41 [16:10] <jcastro> we call this Adopt a Package
  42 [16:10] <jcastro> dholbach: want to go into that for a little bit?
  43 [16:10] <dholbach> sure
  44 [16:11] <jcastro> https://wiki.ubuntu.com/BugSquad/AdoptPackage
  45 [16:11] <dholbach> and https://wiki.ubuntu.com/Upstream/Adopt
  46 [16:11] <dholbach> so to Adopt an Upstream can mean a couple of things
  47 [16:11] <dholbach> the most important one being that you act as a tie betwen Ubuntu and that upstream project
  48 [16:12] <dholbach> you can answer questions for people on that project, you know what's going on there, you know what's going on in Ubuntu
  49 [16:12] <dholbach> an easy example for "knowing what's going on" is: Upstream wants to ship a new feature release 2 weeks before the new Ubuntu release comes out
  50 [16:13] <dholbach> in that case you can talk to them and make it clear that it's hard to get that new feature release into Ubuntu because Ubuntu is already almost frozen
  51 [16:13] <dholbach> you can help with that kind of misinformation and help with the coordination early and explain what to do
  52 [16:13] <dholbach> jcastro and I will talk about the release cycle in more detail later on
  53 [16:14] <dholbach> another thing you can work on is bug reports
  54 [16:14] <dholbach> as we said before Ubuntu exposes great software to millions of users
  55 [16:14] <dholbach> of course these users occasionally run into problems
  56 [16:14] <jcastro> and they might not know what an upstream is!
  57 [16:14] <dholbach> exactly
  58 [16:14] <dholbach> so it's our job to triage those bugs, debug them, get as much information as possible and forward them to the upstream maintainers
  59 [16:15] <dholbach> because they are in the best position to work on the bug effectively
  60 [16:15] <dholbach> if we can fix it that's great, then we'll forward the patch to upstream
  61 [16:15] <jcastro> oh, so we can be good filters you mean?
  62 [16:15] <dholbach> exactly
  63 [16:15] <jcastro> we can weed out all the junk bugs so upstreams can concentrate on the ones with the best debugging info, descriptions, etc.
  64 [16:15] <dholbach> I've heard we have great tools for finding duplicates and dealing with crashes?
  65 [16:16] <dholbach> jcastro: maybe we should explain first what "to triage" means
  66 [16:16] <jcastro> yes.
  67 [16:16] <dholbach> you wanna do it? :)
  68 [16:16] <jcastro> https://wiki.ubuntu.com/Bugs/HowToTriage/
  69 [16:16] <jcastro> sure!
  70 [16:16] <jcastro> so when users report bugs
  71 [16:17] <jcastro> they go into this big pile
  72 [16:17] <jcastro> to triage a bug means to subdivide that pile from one huge unmanageable mess to smaller piles
  73 [16:17] <jcastro> and as the bug continues on in it's lifecycle it will end up in the right pile
  74 [16:17] <jcastro> so, we might start with a big pile
  75 [16:17] <jcastro> but then we have smaller piles, kernel, desktop, printing, etc.
  76 [16:18] <jcastro> and then those get smaller
  77 === tobias is now known as Guest96094
  78 [16:18] <jcastro> until you have a package, like say, nautilus.
  79 [16:18] <jcastro> bug triagers are people who sort the bugs in the right pile
  80 [16:18] <dholbach> then we might be able to get the bug confirmed, make sure we know in which release the bug occurs
  81 [16:18] <jcastro> the great thing about this is that you can do this without being too technical
  82 [16:18] <dholbach> then find the right steps to reproduce it
  83 [16:18] <jcastro> these days we have tools that do a good job of guessing what pile something should go in
  84 [16:19] <jcastro> but we also need to try to reproduce the bug so we can confirm it
  85 [16:19] <jcastro> this involves following what the reporter did
  86 [16:19] <jcastro> so if the report is "clicking on this button breaks printing"
  87 [16:19] <dholbach> https://help.ubuntu.com/community/ReportingBugs has a bit more detail about how this "guessing" works
  88 [16:19] <jcastro> then usually you try it, and see if you see the problem too
  89 [16:19] <jcastro> also, remember that we have millions of users
  90 [16:20] <dholbach> the better we isolate the issue, the less work the upstreams have, they can concentrate on fixing the bug :-)
  91 [16:20] <jcastro> I find that most times bugs sitting in the big pile have already been reported, so you can do a bunch of good just marking things as duplicates (if they are indeed duplicates)
  92 [16:20] <dholbach> right
  93 [16:20] <dholbach> also there's bugs with crash reportsf
  94 === tobias is now known as Guest76403
  95 [16:21] <dholbach> Ubuntu has a fantastic machinery that gets as much debug information about those crashes as possible
  96 [16:21] <dholbach> automatically
  97 [16:21] <jcastro> https://wiki.ubuntu.com/Apport <--- here are pictures of what it looks like
  98 [16:21] <dholbach> so what was a painstaking job before (https://wiki.ubuntu.com/Backtrace) is a lot easier now
  99 [16:22] <dholbach> the end result is that we can say in function do_something() in line 45 of file bla/something.c the whole thing crashed
 100 [16:22] <dholbach> plus other things that developers might want to know :)
 101 [16:22] <dholbach> go and see Emmet Hikory's session on Friday about "Interpreting stacktraces" if you want to know more :)
 102 [16:22] <jcastro> are these important to send to upstreams?
 103 [16:23] <jcastro> they look important!
 104 [16:23] <dholbach> holy cow yes!
 105 [16:23] <dholbach> with all this information it should be a bit easier for upstream developers to find the cause for the crash and fix it
 106 [16:23] <dholbach> a crash in most cases means: the program is gone, it stopped working, unsaved work, etc you know what it's like
 107 [16:24] <dholbach> sometimes these crashes happen in special circumstances that didn't happen for the upstream developers when they tested it
 108 [16:24] <dholbach> in almost every discussion I was part of upstreams were happy about that kind of conversations and grateful for detailed bug reports
 109 [16:24] <dholbach> if you have questions about how to get started with bugs and everything check out https://wiki.ubuntu.com/Upstream/Adopt and talk to the fine people in #ubuntu-bugs
 110 [16:25] <jcastro> ok, so apport finds this and reports this to launchpad, what's the next step?
 111 [16:25] <dholbach> the Launchpad machinery will do its magic and add more debug information to the bug report
 112 [16:25] <dholbach> the next step would be trying to confirm the issue
 113 [16:26] <dholbach> if you can confirm it and maybe add some more details to what happened, you can have a look and see if it was already reports in the upstream bug tracker
 114 [16:26] <dholbach> if it was, see if you can add more information to the bug
 115 [16:26] <dholbach> if it wasn't report it
 116 [16:26] <dholbach> sometimes it's useful to have a look and see if upstreams have some kind of "debugging information" documented somewhere
 117 [16:27] <dholbach> for some of them it will be something like "start the program with --debug option" or something
 118 [16:27] <dholbach> a lot of that kind of information is listed at https://wiki.ubuntu.com/DebuggingProcedures and you should add info for your upstream project if you have any :)
 119 [16:28] <dholbach> also Launchpad has a great feature: you can link the upstream bug to the Ubuntu bug, right?
 120 [16:28] <jcastro> yeah
 121 [16:28] <jcastro> let me show you about that
 122 === godzilla is now known as Guest30844
 123 [16:29] <jcastro> actually, give me a minute to find a bug
 124 [16:29] <jcastro> dholbach: do a dance!
 125 [16:29]  * dholbach puts on some music and does the funky Upstream-Bug-Linking dance
 126 [16:29] <jcastro> https://wiki.ubuntu.com/Bugs/Upstream
 127 [16:30] <jcastro> ok so here is some documentation, depending on the bug
 128 [16:30] <jcastro> however, in general, you can just click "Also affects project" in launchpad, and paste in the bug url from the other bug tracker
 129 [16:30] <jcastro> https://wiki.ubuntu.com/Bugs/Watches
 130 [16:30] <jcastro> they looks like this
 131 [16:30] <jcastro> so you see there, on the top example, there's the  bug in ubuntu, and the bug in the upstream project's bug tracker
 132 [16:30] <jcastro> in this case linux kernel bug 10911
 133 [16:31] <jcastro> these are useful because with certain bug trackers
 134 [16:31] <jcastro> like mozilla's bugzilla, we can do neat things like syncing comments between the 2 bug trackers
 135 [16:32] <jcastro> https://bugs.edge.launchpad.net/ubuntu/+source/notify-osd/+bug/501393
 136 [16:32] <jcastro> here is a good example of both ubuntu and mozilla developers working on a bug for example
 137 [16:32] <jcastro> however, just linking the bug can help
 138 [16:32] <jcastro> because it gives everyone a place to gather discussion around a bug
 139 [16:33] <jcastro> and especially for new users that might not know how things work
 140 [16:33] <jcastro> what other information can we help upstreams with dholbach?
 141 [16:33] <jcastro> tell me about this schedule
 142 [16:33] <dholbach> another fine thing you can do is: plan a Bug Day with your Upstream: invite them to https://wiki.ubuntu.com/UbuntuBugDay - triage bugs together, give people some coaching with that particular software - once you're good friends with your upstream, sky's the limit
 143 [16:34] <dholbach> alright... release schedule!
 144 [16:34] <jcastro> upstreams ask me all the time about the schedule
 145 [16:34] <jcastro> but sometimes I don't know what means what!
 146 [16:34] <dholbach> have a look at https://wiki.ubuntu.com/ReleaseSchedule
 147 [16:34] <dholbach> it shows our current release cycle
 148 [16:35] <dholbach> you can see how it goes from green in the beginning of the 6 months cycle to red at the end
 149 [16:35] <dholbach> green doesn't mean "it's all good, everything works" :-)
 150 [16:35] <dholbach> it rather means "everything's allowed from a developer point of view"
 151 [16:36] <dholbach> so we start with setting up the build tools in the beginning, plan features at UDS and merge with Debian (more on merges, syncs and Debian in a bit :-))
 152 [16:37] <dholbach> at some stage we hit feature freeze, that's where we plan to have most of the feature in the distro
 153 [16:37] <jcastro> Xazax | why is the release cycle for upstream projects important?
 154 [16:37] <dholbach> doesn't mean they have to be all perfect
 155 [16:37] <dholbach> Xazax: will get to that in a sec
 156 [16:37] <dholbach> but it means that they have to be there and be test-able :)
 157 [16:38] <dholbach> so we move from feature development to fixing
 158 [16:38] <dholbach> and stabilisation
 159 [16:38] <dholbach> the later it gets in the release cycle the more conservative we get
 160 [16:38] <dholbach> to upload a small patch that fixes a crash 3 weeks before release is probably fine
 161 [16:38] <dholbach> but to upload a GIANT new release with n+1 new features will probably get you in trouble with the release managers
 162 [16:39] <dholbach> Jorge and I worked on a cheatsheet that makes it easier to understand what is allowed at which stage of the release cycle to ease the planning and coordination somewhat: http://people.canonical.com/~dholbach/cheatsheet.pdf
 163 [16:39] <dholbach> that's linked from https://wiki.ubuntu.com/Upstream/Adopt too
 164 [16:39] <dholbach> I hope that answers Xazax's question somewhat.
 165 [16:40] <jcastro> i would like to chime in on his question
 166 [16:40] <dholbach> sure
 167 [16:40] <jcastro> with things like a Long Term Support release it's critical for us as a downstream distro to communicate our release schedule
 168 [16:40] <jcastro> we sync our schedule with GNOME
 169 [16:40] <jcastro> so, we're kind of lined up
 170 [16:41] <jcastro> some upstreams want to know when they should release to have their software in ubuntu
 171 [16:41] <jcastro> so they can make a release, or decided to skip an ubuntu release, etc.
 172 [16:41] <jcastro> so, when we're at UDS
 173 [16:41] <jcastro> and we're talking to upstreams we make decisions on things dependent on release cycles
 174 [16:42] <jcastro> so it might be "well, that feature won't be ready in time, so we'll go with version 1.x instead" or something
 175 [16:42] <dholbach> that sounds very reasonable
 176 [16:42] <dholbach> whatever we can't fix during the release time, we have to fix after the release... jcastro: do you know how that works?
 177 [16:42] <jcastro> yes
 178 [16:43] <jcastro> it's a big source of confusion
 179 [16:43] <jcastro> so let's talk about Stable Release Updates
 180 [16:43] <dholbach> I think it all started with something like this: http://blog.somekool.net/images/img_4232.jpg
 181 [16:43] <jcastro> https://wiki.ubuntu.com/StableReleaseUpdates
 182 [16:43] <jcastro> doh!
 183 [16:43] <dholbach> shall I tell the story quickly before you move on?
 184 [16:43] <jcastro> yes!
 185 [16:43] <dholbach> ok
 186 [16:44] <dholbach> so in the very early days of Ubuntu we uploaded a couple of fixes (they all were upstream already) to the xserver to a stable release
 187 [16:44] <dholbach> that means that users who had installed the release (and not the development version) got those fixes
 188 [16:44] <dholbach> the developer had tested them all, they were upstream fixes and everything seemed good
 189 [16:45] <dholbach> the problem was that a certain percentage of our users (x% of some millions) suddenly had a broken Xserver
 190 [16:45] <dholbach> which resulted in people like my dad having to wrestle with: http://blog.somekool.net/images/img_4232.jpg
 191 [16:45] <dholbach> so we realised that we need to be more conservative with fixes like that
 192 [16:46] <dholbach> that's where SRUs originate from
 193 [16:46] <dholbach> jcastro: so how does this all work now?
 194 [16:46] <jcastro> https://wiki.ubuntu.com/StableReleaseUpdates
 195 [16:46] <jcastro> we have these
 196 [16:47] <jcastro> don't worry, it only looks scary!
 197 [16:47] <jcastro> the key point to remember here when talking to upstreams
 198 [16:47] <jcastro> is that really bad bugs should be communicated to distros
 199 [16:47] <jcastro> you see this all the time on like planet gnome
 200 [16:48] <jcastro> "wow, we found a bad bug in gstreamer, distros should ship this fix: $url" or something
 201 [16:48] <jcastro> as you hang out and talk with upstreams
 202 [16:48] <jcastro> at some point they will have a critical bug that will need to be fixed
 203 [16:48] <jcastro> so usually you can take ownership of that, talk to the ubuntu maintainer, and communicate the needs that way
 204 [16:48] <jcastro> sometimes upstreams fix small bugs
 205 [16:48] <kfogel> dhasenan: thanks
 206 [16:49] <jcastro> and of course those are good
 207 [16:49] <kfogel> whhooops
 208 [16:49] <kfogel> dholbach: thanks
 209 [16:49]  * kfogel watches out for nick completion
 210 [16:49] <jcastro> but it can be frustrating to an upstream when the fix doesn't get into an SRU
 211 [16:49] <jcastro> it's just one of those things where the fix needs to be weighed with the risk
 212 [16:50] <dholbach> https://wiki.ubuntu.com/StableReleaseUpdates is pretty good at explaining what kind of fixes we want and which make sense
 213 [16:50] <jcastro> In general I think we do a good job here, but if an upstream still wants new crack in the distro, we have things like PPAs and backports
 214 [16:50] <jcastro> that's also an area where you can contribute
 215 [16:50] <jcastro> many ubuntu contributors set up and run PPAs as a service to upstreams for apps people love
 216 [16:50] <dholbach> http://people.canonical.com/~dholbach/cheatsheet.pdf should give you an overview over what to do in which case
 217 [16:51] <jcastro> that is highly encouraged, and a good way to learn packaging if you want to start the  road to becoming an ubuntu developer
 218 [16:51] <jcastro> we only have 5 minutes left, so let's talk about debian for  a bit
 219 [16:52] <dholbach> one of our most important upstreams is Debian
 220 [16:52] <dholbach> and we inherit a lot of packages and lots of great work from Debian developers
 221 [16:52] <dholbach> usually it works like this: if a package is unmodified in Ubuntu and we're before DebianImportFreeze (Check the release schedule again), it gets "synced" automatically
 222 [16:53] <dholbach> which means that the package in Ubuntu will get overwritten with the source that is in Debian right now (and rebuilt)
 223 === zen is now known as Guest90818
 224 [16:53] <dholbach> if it's modified in Ubuntu, we need to figure out if we can drop out changes (then we'll ask for a sync)
 225 [16:53] <dholbach> and if we can't drop our changes, we need to merge the changes semi-manually
 226 [16:54] <dholbach> https://wiki.ubuntu.com/UbuntuDevelopment/Merging explains how the merging works and which tools we use
 227 [16:54] <dholbach> common scenarios for differences between Ubuntu and Debian are:
 228 [16:54] <dholbach>  - we're in different parts of our release cycle and we decide to ship a newer version
 229 [16:55] <dholbach>  - new improvements we want to try out in Ubuntu (launchpad integration of gnome apps, new indicators, new notifications, etc.)
 230 [16:55] <dholbach> and lots of other small things because of different things we support and not support
 231 [16:55] <dholbach> but generally the idea is: the more we can have in sync with Debian, the better
 232 [16:55] <dholbach> it means less merging, less maintenance hassle and a common code base to look at
 233 [16:56] <dholbach> so whatever is suitable in terms of patches to forward to Debian, we do forward
 234 [16:56] <dholbach> https://wiki.ubuntu.com/Debian/Bugs has more details on that
 235 [16:56] <jcastro> whew
 236 [16:56] <jcastro> that was alot of info!
 237 [16:56] <jcastro> and only one question, so either we were really great or really bad!
 238 [16:57] <jcastro> \o/
 239 [16:57] <dholbach> that's the great thing about working on a certain piece of software is you get to know: lots of users (and make them happy with fixes), the upstream maintainers, lots of great Ubuntu developers that work with you and maintainers of others distros too
 240 [16:57] <dholbach> you not only make the world a better place by working on making software better for everybody, but you also get to know a lot of great people AND learn something new :)
 241 [16:58] <dholbach> one page you should definitely bookmark is: https://wiki.ubuntu.com/Upstream/Adopt :)
 242 [16:58] <dholbach> any final questions?
 243 [16:58] <dholbach> jcastro: any final words? :)
 244 [16:58] <jcastro> nope, thanks for coming everyone!
 245 [16:59] <dholbach> thanks everybody - you ROCK!