Dev Week -- Adopt-an-Upstream -- jcastro and dholbach -- Thu, Jan 28
1 [16:01] <dholbach> I'm here with the amazing Jorge O. Castro, jcastro! 2 [16:01] <jcastro> woo hoo! 3 [16:01] <jcastro> we have been preparing real hard for this session, hope you like it! 4 === godzilla is now known as Guest53616 5 [16:02] <dholbach> And together we'll talk about Adopting an Upstream! Why it's fun, necessary and makes the world a better place. 6 [16:02] <dholbach> jcastro: What's an Upstream? 7 [16:02] <jcastro> that's a good question 8 [16:02] <jcastro> first, when you get an ubuntu CD it comes with a bunch of great software 9 [16:02] <jcastro> things like Firefox, Openoffice, GNOME, KDE, etc. 10 [16:03] <jcastro> what Ubuntu does as a distribution is take all those things, assemble them into one operating system, and then ship 11 [16:03] <jcastro> so, these projects, we call them upstream, like in a river 12 [16:03] <jcastro> and distributions like us are downstram 13 [16:03] <jcastro> and derived distributions like Mint would be downstream from us 14 [16:03] <dholbach> so that's like all the software authors of those projects? 15 [16:03] <jcastro> working with upstreams and downstreams is a critical part of ubuntu 16 [16:03] <jcastro> right! 17 [16:03] <dholbach> (I hope I encourage YOU ALL to ask questions too, I asked the first ones myself... :-)) 18 [16:04] <jcastro> so, for example, it's critical for us to be able to work with upstreams like Firefox to ensure that we're shipping something users will love 19 [16:04] <jcastro> so for example this week in #ubuntu-mozillateam you'll see our Ubuntu teams working with people from Mozilla 20 [16:04] <dholbach> So there's millions of Ubuntu users and there's thousands of Upstream projects... looks like being there for both huge groups is quite an undertaking 21 [16:04] <jcastro> so there's a bunch of cross boundary work going on there that some people don't see 22 [16:05] <jcastro> right, especially when you're trying to fix a problem! But we'll show you how to do that when we get there 23 [16:05] <jcastro> you'll see that sometimes putting a bug in the right place that we can be a service to upstreams when it comes to testing. 24 [16:06] <jcastro> and when we fix problems it's of course our duty to make sure that those fixes get back upstream so that they can improve the overall quality of their project 25 [16:06] <jcastro> and of course, the other half is ensuring that when upstream fixes bugs that we get those out to users, because at the end of the day that's what it's all about 26 [16:06] <jcastro> so 27 [16:06] <jcastro> I have been working on some wiki pages to help people get started 28 [16:07] <jcastro> https://wiki.ubuntu.com/Upstream 29 [16:07] <jcastro> It's unreasonable to expect every upstream author to know in detail how ubuntu works 30 [16:07] <jcastro> some are content just making their software 31 [16:07] <dholbach> and the other way around :) 32 [16:08] <jcastro> so I've put together a little cheat sheet here for them (you can help improve this too if you find mistakes) 33 [16:08] <jcastro> so that they can have one place to find important things, like when we freeze, what our security policy is, etc. 34 [16:08] <jcastro> and also give them information on how things are going on the ubuntu platform 35 [16:08] <jcastro> so if you wanted to find out about notify osd or messaging indicators, I've listed those there 36 [16:09] <jcastro> now 37 [16:09] <jcastro> one thing we've all been working on is a little program to give upstreams an "in" in the distro 38 [16:09] <jcastro> this would be a person that does know how Ubuntu works 39 [16:09] <jcastro> and is passionate about that specific piece of software 40 [16:10] <jcastro> and wants to work on it in Ubuntu but also be there for the upstreams to answer their questions, be an information source, and be there when things go wrong 41 [16:10] <jcastro> we call this Adopt a Package 42 [16:10] <jcastro> dholbach: want to go into that for a little bit? 43 [16:10] <dholbach> sure 44 [16:11] <jcastro> https://wiki.ubuntu.com/BugSquad/AdoptPackage 45 [16:11] <dholbach> and https://wiki.ubuntu.com/Upstream/Adopt 46 [16:11] <dholbach> so to Adopt an Upstream can mean a couple of things 47 [16:11] <dholbach> the most important one being that you act as a tie betwen Ubuntu and that upstream project 48 [16:12] <dholbach> you can answer questions for people on that project, you know what's going on there, you know what's going on in Ubuntu 49 [16:12] <dholbach> an easy example for "knowing what's going on" is: Upstream wants to ship a new feature release 2 weeks before the new Ubuntu release comes out 50 [16:13] <dholbach> in that case you can talk to them and make it clear that it's hard to get that new feature release into Ubuntu because Ubuntu is already almost frozen 51 [16:13] <dholbach> you can help with that kind of misinformation and help with the coordination early and explain what to do 52 [16:13] <dholbach> jcastro and I will talk about the release cycle in more detail later on 53 [16:14] <dholbach> another thing you can work on is bug reports 54 [16:14] <dholbach> as we said before Ubuntu exposes great software to millions of users 55 [16:14] <dholbach> of course these users occasionally run into problems 56 [16:14] <jcastro> and they might not know what an upstream is! 57 [16:14] <dholbach> exactly 58 [16:14] <dholbach> so it's our job to triage those bugs, debug them, get as much information as possible and forward them to the upstream maintainers 59 [16:15] <dholbach> because they are in the best position to work on the bug effectively 60 [16:15] <dholbach> if we can fix it that's great, then we'll forward the patch to upstream 61 [16:15] <jcastro> oh, so we can be good filters you mean? 62 [16:15] <dholbach> exactly 63 [16:15] <jcastro> we can weed out all the junk bugs so upstreams can concentrate on the ones with the best debugging info, descriptions, etc. 64 [16:15] <dholbach> I've heard we have great tools for finding duplicates and dealing with crashes? 65 [16:16] <dholbach> jcastro: maybe we should explain first what "to triage" means 66 [16:16] <jcastro> yes. 67 [16:16] <dholbach> you wanna do it? :) 68 [16:16] <jcastro> https://wiki.ubuntu.com/Bugs/HowToTriage/ 69 [16:16] <jcastro> sure! 70 [16:16] <jcastro> so when users report bugs 71 [16:17] <jcastro> they go into this big pile 72 [16:17] <jcastro> to triage a bug means to subdivide that pile from one huge unmanageable mess to smaller piles 73 [16:17] <jcastro> and as the bug continues on in it's lifecycle it will end up in the right pile 74 [16:17] <jcastro> so, we might start with a big pile 75 [16:17] <jcastro> but then we have smaller piles, kernel, desktop, printing, etc. 76 [16:18] <jcastro> and then those get smaller 77 === tobias is now known as Guest96094 78 [16:18] <jcastro> until you have a package, like say, nautilus. 79 [16:18] <jcastro> bug triagers are people who sort the bugs in the right pile 80 [16:18] <dholbach> then we might be able to get the bug confirmed, make sure we know in which release the bug occurs 81 [16:18] <jcastro> the great thing about this is that you can do this without being too technical 82 [16:18] <dholbach> then find the right steps to reproduce it 83 [16:18] <jcastro> these days we have tools that do a good job of guessing what pile something should go in 84 [16:19] <jcastro> but we also need to try to reproduce the bug so we can confirm it 85 [16:19] <jcastro> this involves following what the reporter did 86 [16:19] <jcastro> so if the report is "clicking on this button breaks printing" 87 [16:19] <dholbach> https://help.ubuntu.com/community/ReportingBugs has a bit more detail about how this "guessing" works 88 [16:19] <jcastro> then usually you try it, and see if you see the problem too 89 [16:19] <jcastro> also, remember that we have millions of users 90 [16:20] <dholbach> the better we isolate the issue, the less work the upstreams have, they can concentrate on fixing the bug :-) 91 [16:20] <jcastro> I find that most times bugs sitting in the big pile have already been reported, so you can do a bunch of good just marking things as duplicates (if they are indeed duplicates) 92 [16:20] <dholbach> right 93 [16:20] <dholbach> also there's bugs with crash reportsf 94 === tobias is now known as Guest76403 95 [16:21] <dholbach> Ubuntu has a fantastic machinery that gets as much debug information about those crashes as possible 96 [16:21] <dholbach> automatically 97 [16:21] <jcastro> https://wiki.ubuntu.com/Apport <--- here are pictures of what it looks like 98 [16:21] <dholbach> so what was a painstaking job before (https://wiki.ubuntu.com/Backtrace) is a lot easier now 99 [16:22] <dholbach> the end result is that we can say in function do_something() in line 45 of file bla/something.c the whole thing crashed 100 [16:22] <dholbach> plus other things that developers might want to know :) 101 [16:22] <dholbach> go and see Emmet Hikory's session on Friday about "Interpreting stacktraces" if you want to know more :) 102 [16:22] <jcastro> are these important to send to upstreams? 103 [16:23] <jcastro> they look important! 104 [16:23] <dholbach> holy cow yes! 105 [16:23] <dholbach> with all this information it should be a bit easier for upstream developers to find the cause for the crash and fix it 106 [16:23] <dholbach> a crash in most cases means: the program is gone, it stopped working, unsaved work, etc you know what it's like 107 [16:24] <dholbach> sometimes these crashes happen in special circumstances that didn't happen for the upstream developers when they tested it 108 [16:24] <dholbach> in almost every discussion I was part of upstreams were happy about that kind of conversations and grateful for detailed bug reports 109 [16:24] <dholbach> if you have questions about how to get started with bugs and everything check out https://wiki.ubuntu.com/Upstream/Adopt and talk to the fine people in #ubuntu-bugs 110 [16:25] <jcastro> ok, so apport finds this and reports this to launchpad, what's the next step? 111 [16:25] <dholbach> the Launchpad machinery will do its magic and add more debug information to the bug report 112 [16:25] <dholbach> the next step would be trying to confirm the issue 113 [16:26] <dholbach> if you can confirm it and maybe add some more details to what happened, you can have a look and see if it was already reports in the upstream bug tracker 114 [16:26] <dholbach> if it was, see if you can add more information to the bug 115 [16:26] <dholbach> if it wasn't report it 116 [16:26] <dholbach> sometimes it's useful to have a look and see if upstreams have some kind of "debugging information" documented somewhere 117 [16:27] <dholbach> for some of them it will be something like "start the program with --debug option" or something 118 [16:27] <dholbach> a lot of that kind of information is listed at https://wiki.ubuntu.com/DebuggingProcedures and you should add info for your upstream project if you have any :) 119 [16:28] <dholbach> also Launchpad has a great feature: you can link the upstream bug to the Ubuntu bug, right? 120 [16:28] <jcastro> yeah 121 [16:28] <jcastro> let me show you about that 122 === godzilla is now known as Guest30844 123 [16:29] <jcastro> actually, give me a minute to find a bug 124 [16:29] <jcastro> dholbach: do a dance! 125 [16:29] * dholbach puts on some music and does the funky Upstream-Bug-Linking dance 126 [16:29] <jcastro> https://wiki.ubuntu.com/Bugs/Upstream 127 [16:30] <jcastro> ok so here is some documentation, depending on the bug 128 [16:30] <jcastro> however, in general, you can just click "Also affects project" in launchpad, and paste in the bug url from the other bug tracker 129 [16:30] <jcastro> https://wiki.ubuntu.com/Bugs/Watches 130 [16:30] <jcastro> they looks like this 131 [16:30] <jcastro> so you see there, on the top example, there's the bug in ubuntu, and the bug in the upstream project's bug tracker 132 [16:30] <jcastro> in this case linux kernel bug 10911 133 [16:31] <jcastro> these are useful because with certain bug trackers 134 [16:31] <jcastro> like mozilla's bugzilla, we can do neat things like syncing comments between the 2 bug trackers 135 [16:32] <jcastro> https://bugs.edge.launchpad.net/ubuntu/+source/notify-osd/+bug/501393 136 [16:32] <jcastro> here is a good example of both ubuntu and mozilla developers working on a bug for example 137 [16:32] <jcastro> however, just linking the bug can help 138 [16:32] <jcastro> because it gives everyone a place to gather discussion around a bug 139 [16:33] <jcastro> and especially for new users that might not know how things work 140 [16:33] <jcastro> what other information can we help upstreams with dholbach? 141 [16:33] <jcastro> tell me about this schedule 142 [16:33] <dholbach> another fine thing you can do is: plan a Bug Day with your Upstream: invite them to https://wiki.ubuntu.com/UbuntuBugDay - triage bugs together, give people some coaching with that particular software - once you're good friends with your upstream, sky's the limit 143 [16:34] <dholbach> alright... release schedule! 144 [16:34] <jcastro> upstreams ask me all the time about the schedule 145 [16:34] <jcastro> but sometimes I don't know what means what! 146 [16:34] <dholbach> have a look at https://wiki.ubuntu.com/ReleaseSchedule 147 [16:34] <dholbach> it shows our current release cycle 148 [16:35] <dholbach> you can see how it goes from green in the beginning of the 6 months cycle to red at the end 149 [16:35] <dholbach> green doesn't mean "it's all good, everything works" :-) 150 [16:35] <dholbach> it rather means "everything's allowed from a developer point of view" 151 [16:36] <dholbach> so we start with setting up the build tools in the beginning, plan features at UDS and merge with Debian (more on merges, syncs and Debian in a bit :-)) 152 [16:37] <dholbach> at some stage we hit feature freeze, that's where we plan to have most of the feature in the distro 153 [16:37] <jcastro> Xazax | why is the release cycle for upstream projects important? 154 [16:37] <dholbach> doesn't mean they have to be all perfect 155 [16:37] <dholbach> Xazax: will get to that in a sec 156 [16:37] <dholbach> but it means that they have to be there and be test-able :) 157 [16:38] <dholbach> so we move from feature development to fixing 158 [16:38] <dholbach> and stabilisation 159 [16:38] <dholbach> the later it gets in the release cycle the more conservative we get 160 [16:38] <dholbach> to upload a small patch that fixes a crash 3 weeks before release is probably fine 161 [16:38] <dholbach> but to upload a GIANT new release with n+1 new features will probably get you in trouble with the release managers 162 [16:39] <dholbach> Jorge and I worked on a cheatsheet that makes it easier to understand what is allowed at which stage of the release cycle to ease the planning and coordination somewhat: http://people.canonical.com/~dholbach/cheatsheet.pdf 163 [16:39] <dholbach> that's linked from https://wiki.ubuntu.com/Upstream/Adopt too 164 [16:39] <dholbach> I hope that answers Xazax's question somewhat. 165 [16:40] <jcastro> i would like to chime in on his question 166 [16:40] <dholbach> sure 167 [16:40] <jcastro> with things like a Long Term Support release it's critical for us as a downstream distro to communicate our release schedule 168 [16:40] <jcastro> we sync our schedule with GNOME 169 [16:40] <jcastro> so, we're kind of lined up 170 [16:41] <jcastro> some upstreams want to know when they should release to have their software in ubuntu 171 [16:41] <jcastro> so they can make a release, or decided to skip an ubuntu release, etc. 172 [16:41] <jcastro> so, when we're at UDS 173 [16:41] <jcastro> and we're talking to upstreams we make decisions on things dependent on release cycles 174 [16:42] <jcastro> so it might be "well, that feature won't be ready in time, so we'll go with version 1.x instead" or something 175 [16:42] <dholbach> that sounds very reasonable 176 [16:42] <dholbach> whatever we can't fix during the release time, we have to fix after the release... jcastro: do you know how that works? 177 [16:42] <jcastro> yes 178 [16:43] <jcastro> it's a big source of confusion 179 [16:43] <jcastro> so let's talk about Stable Release Updates 180 [16:43] <dholbach> I think it all started with something like this: http://blog.somekool.net/images/img_4232.jpg 181 [16:43] <jcastro> https://wiki.ubuntu.com/StableReleaseUpdates 182 [16:43] <jcastro> doh! 183 [16:43] <dholbach> shall I tell the story quickly before you move on? 184 [16:43] <jcastro> yes! 185 [16:43] <dholbach> ok 186 [16:44] <dholbach> so in the very early days of Ubuntu we uploaded a couple of fixes (they all were upstream already) to the xserver to a stable release 187 [16:44] <dholbach> that means that users who had installed the release (and not the development version) got those fixes 188 [16:44] <dholbach> the developer had tested them all, they were upstream fixes and everything seemed good 189 [16:45] <dholbach> the problem was that a certain percentage of our users (x% of some millions) suddenly had a broken Xserver 190 [16:45] <dholbach> which resulted in people like my dad having to wrestle with: http://blog.somekool.net/images/img_4232.jpg 191 [16:45] <dholbach> so we realised that we need to be more conservative with fixes like that 192 [16:46] <dholbach> that's where SRUs originate from 193 [16:46] <dholbach> jcastro: so how does this all work now? 194 [16:46] <jcastro> https://wiki.ubuntu.com/StableReleaseUpdates 195 [16:46] <jcastro> we have these 196 [16:47] <jcastro> don't worry, it only looks scary! 197 [16:47] <jcastro> the key point to remember here when talking to upstreams 198 [16:47] <jcastro> is that really bad bugs should be communicated to distros 199 [16:47] <jcastro> you see this all the time on like planet gnome 200 [16:48] <jcastro> "wow, we found a bad bug in gstreamer, distros should ship this fix: $url" or something 201 [16:48] <jcastro> as you hang out and talk with upstreams 202 [16:48] <jcastro> at some point they will have a critical bug that will need to be fixed 203 [16:48] <jcastro> so usually you can take ownership of that, talk to the ubuntu maintainer, and communicate the needs that way 204 [16:48] <jcastro> sometimes upstreams fix small bugs 205 [16:48] <kfogel> dhasenan: thanks 206 [16:49] <jcastro> and of course those are good 207 [16:49] <kfogel> whhooops 208 [16:49] <kfogel> dholbach: thanks 209 [16:49] * kfogel watches out for nick completion 210 [16:49] <jcastro> but it can be frustrating to an upstream when the fix doesn't get into an SRU 211 [16:49] <jcastro> it's just one of those things where the fix needs to be weighed with the risk 212 [16:50] <dholbach> https://wiki.ubuntu.com/StableReleaseUpdates is pretty good at explaining what kind of fixes we want and which make sense 213 [16:50] <jcastro> In general I think we do a good job here, but if an upstream still wants new crack in the distro, we have things like PPAs and backports 214 [16:50] <jcastro> that's also an area where you can contribute 215 [16:50] <jcastro> many ubuntu contributors set up and run PPAs as a service to upstreams for apps people love 216 [16:50] <dholbach> http://people.canonical.com/~dholbach/cheatsheet.pdf should give you an overview over what to do in which case 217 [16:51] <jcastro> that is highly encouraged, and a good way to learn packaging if you want to start the road to becoming an ubuntu developer 218 [16:51] <jcastro> we only have 5 minutes left, so let's talk about debian for a bit 219 [16:52] <dholbach> one of our most important upstreams is Debian 220 [16:52] <dholbach> and we inherit a lot of packages and lots of great work from Debian developers 221 [16:52] <dholbach> usually it works like this: if a package is unmodified in Ubuntu and we're before DebianImportFreeze (Check the release schedule again), it gets "synced" automatically 222 [16:53] <dholbach> which means that the package in Ubuntu will get overwritten with the source that is in Debian right now (and rebuilt) 223 === zen is now known as Guest90818 224 [16:53] <dholbach> if it's modified in Ubuntu, we need to figure out if we can drop out changes (then we'll ask for a sync) 225 [16:53] <dholbach> and if we can't drop our changes, we need to merge the changes semi-manually 226 [16:54] <dholbach> https://wiki.ubuntu.com/UbuntuDevelopment/Merging explains how the merging works and which tools we use 227 [16:54] <dholbach> common scenarios for differences between Ubuntu and Debian are: 228 [16:54] <dholbach> - we're in different parts of our release cycle and we decide to ship a newer version 229 [16:55] <dholbach> - new improvements we want to try out in Ubuntu (launchpad integration of gnome apps, new indicators, new notifications, etc.) 230 [16:55] <dholbach> and lots of other small things because of different things we support and not support 231 [16:55] <dholbach> but generally the idea is: the more we can have in sync with Debian, the better 232 [16:55] <dholbach> it means less merging, less maintenance hassle and a common code base to look at 233 [16:56] <dholbach> so whatever is suitable in terms of patches to forward to Debian, we do forward 234 [16:56] <dholbach> https://wiki.ubuntu.com/Debian/Bugs has more details on that 235 [16:56] <jcastro> whew 236 [16:56] <jcastro> that was alot of info! 237 [16:56] <jcastro> and only one question, so either we were really great or really bad! 238 [16:57] <jcastro> \o/ 239 [16:57] <dholbach> that's the great thing about working on a certain piece of software is you get to know: lots of users (and make them happy with fixes), the upstream maintainers, lots of great Ubuntu developers that work with you and maintainers of others distros too 240 [16:57] <dholbach> you not only make the world a better place by working on making software better for everybody, but you also get to know a lot of great people AND learn something new :) 241 [16:58] <dholbach> one page you should definitely bookmark is: https://wiki.ubuntu.com/Upstream/Adopt :) 242 [16:58] <dholbach> any final questions? 243 [16:58] <dholbach> jcastro: any final words? :) 244 [16:58] <jcastro> nope, thanks for coming everyone! 245 [16:59] <dholbach> thanks everybody - you ROCK!