## page was renamed from MeetingLogs/devweek1001/devweek1001/AdoptUpstream == Dev Week -- Adopt-an-Upstream -- jcastro and dholbach -- Thu, Jan 28 == UTC {{{#!IRC [16:01] I'm here with the amazing Jorge O. Castro, jcastro! [16:01] woo hoo! [16:01] we have been preparing real hard for this session, hope you like it! === godzilla is now known as Guest53616 [16:02] And together we'll talk about Adopting an Upstream! Why it's fun, necessary and makes the world a better place. [16:02] jcastro: What's an Upstream? [16:02] that's a good question [16:02] first, when you get an ubuntu CD it comes with a bunch of great software [16:02] things like Firefox, Openoffice, GNOME, KDE, etc. [16:03] what Ubuntu does as a distribution is take all those things, assemble them into one operating system, and then ship [16:03] so, these projects, we call them upstream, like in a river [16:03] and distributions like us are downstram [16:03] and derived distributions like Mint would be downstream from us [16:03] so that's like all the software authors of those projects? [16:03] working with upstreams and downstreams is a critical part of ubuntu [16:03] right! [16:03] (I hope I encourage YOU ALL to ask questions too, I asked the first ones myself... :-)) [16:04] so, for example, it's critical for us to be able to work with upstreams like Firefox to ensure that we're shipping something users will love [16:04] so for example this week in #ubuntu-mozillateam you'll see our Ubuntu teams working with people from Mozilla [16:04] So there's millions of Ubuntu users and there's thousands of Upstream projects... looks like being there for both huge groups is quite an undertaking [16:04] so there's a bunch of cross boundary work going on there that some people don't see [16:05] right, especially when you're trying to fix a problem! But we'll show you how to do that when we get there [16:05] you'll see that sometimes putting a bug in the right place that we can be a service to upstreams when it comes to testing. [16:06] and when we fix problems it's of course our duty to make sure that those fixes get back upstream so that they can improve the overall quality of their project [16:06] and of course, the other half is ensuring that when upstream fixes bugs that we get those out to users, because at the end of the day that's what it's all about [16:06] so [16:06] I have been working on some wiki pages to help people get started [16:07] https://wiki.ubuntu.com/Upstream [16:07] It's unreasonable to expect every upstream author to know in detail how ubuntu works [16:07] some are content just making their software [16:07] and the other way around :) [16:08] so I've put together a little cheat sheet here for them (you can help improve this too if you find mistakes) [16:08] so that they can have one place to find important things, like when we freeze, what our security policy is, etc. [16:08] and also give them information on how things are going on the ubuntu platform [16:08] so if you wanted to find out about notify osd or messaging indicators, I've listed those there [16:09] now [16:09] one thing we've all been working on is a little program to give upstreams an "in" in the distro [16:09] this would be a person that does know how Ubuntu works [16:09] and is passionate about that specific piece of software [16:10] and wants to work on it in Ubuntu but also be there for the upstreams to answer their questions, be an information source, and be there when things go wrong [16:10] we call this Adopt a Package [16:10] dholbach: want to go into that for a little bit? [16:10] sure [16:11] https://wiki.ubuntu.com/BugSquad/AdoptPackage [16:11] and https://wiki.ubuntu.com/Upstream/Adopt [16:11] so to Adopt an Upstream can mean a couple of things [16:11] the most important one being that you act as a tie betwen Ubuntu and that upstream project [16:12] you can answer questions for people on that project, you know what's going on there, you know what's going on in Ubuntu [16:12] an easy example for "knowing what's going on" is: Upstream wants to ship a new feature release 2 weeks before the new Ubuntu release comes out [16:13] in that case you can talk to them and make it clear that it's hard to get that new feature release into Ubuntu because Ubuntu is already almost frozen [16:13] you can help with that kind of misinformation and help with the coordination early and explain what to do [16:13] jcastro and I will talk about the release cycle in more detail later on [16:14] another thing you can work on is bug reports [16:14] as we said before Ubuntu exposes great software to millions of users [16:14] of course these users occasionally run into problems [16:14] and they might not know what an upstream is! [16:14] exactly [16:14] so it's our job to triage those bugs, debug them, get as much information as possible and forward them to the upstream maintainers [16:15] because they are in the best position to work on the bug effectively [16:15] if we can fix it that's great, then we'll forward the patch to upstream [16:15] oh, so we can be good filters you mean? [16:15] exactly [16:15] we can weed out all the junk bugs so upstreams can concentrate on the ones with the best debugging info, descriptions, etc. [16:15] I've heard we have great tools for finding duplicates and dealing with crashes? [16:16] jcastro: maybe we should explain first what "to triage" means [16:16] yes. [16:16] you wanna do it? :) [16:16] https://wiki.ubuntu.com/Bugs/HowToTriage/ [16:16] sure! [16:16] so when users report bugs [16:17] they go into this big pile [16:17] to triage a bug means to subdivide that pile from one huge unmanageable mess to smaller piles [16:17] and as the bug continues on in it's lifecycle it will end up in the right pile [16:17] so, we might start with a big pile [16:17] but then we have smaller piles, kernel, desktop, printing, etc. [16:18] and then those get smaller === tobias is now known as Guest96094 [16:18] until you have a package, like say, nautilus. [16:18] bug triagers are people who sort the bugs in the right pile [16:18] then we might be able to get the bug confirmed, make sure we know in which release the bug occurs [16:18] the great thing about this is that you can do this without being too technical [16:18] then find the right steps to reproduce it [16:18] these days we have tools that do a good job of guessing what pile something should go in [16:19] but we also need to try to reproduce the bug so we can confirm it [16:19] this involves following what the reporter did [16:19] so if the report is "clicking on this button breaks printing" [16:19] https://help.ubuntu.com/community/ReportingBugs has a bit more detail about how this "guessing" works [16:19] then usually you try it, and see if you see the problem too [16:19] also, remember that we have millions of users [16:20] the better we isolate the issue, the less work the upstreams have, they can concentrate on fixing the bug :-) [16:20] I find that most times bugs sitting in the big pile have already been reported, so you can do a bunch of good just marking things as duplicates (if they are indeed duplicates) [16:20] right [16:20] also there's bugs with crash reportsf === tobias is now known as Guest76403 [16:21] Ubuntu has a fantastic machinery that gets as much debug information about those crashes as possible [16:21] automatically [16:21] https://wiki.ubuntu.com/Apport <--- here are pictures of what it looks like [16:21] so what was a painstaking job before (https://wiki.ubuntu.com/Backtrace) is a lot easier now [16:22] the end result is that we can say in function do_something() in line 45 of file bla/something.c the whole thing crashed [16:22] plus other things that developers might want to know :) [16:22] go and see Emmet Hikory's session on Friday about "Interpreting stacktraces" if you want to know more :) [16:22] are these important to send to upstreams? [16:23] they look important! [16:23] holy cow yes! [16:23] with all this information it should be a bit easier for upstream developers to find the cause for the crash and fix it [16:23] a crash in most cases means: the program is gone, it stopped working, unsaved work, etc you know what it's like [16:24] sometimes these crashes happen in special circumstances that didn't happen for the upstream developers when they tested it [16:24] in almost every discussion I was part of upstreams were happy about that kind of conversations and grateful for detailed bug reports [16:24] if you have questions about how to get started with bugs and everything check out https://wiki.ubuntu.com/Upstream/Adopt and talk to the fine people in #ubuntu-bugs [16:25] ok, so apport finds this and reports this to launchpad, what's the next step? [16:25] the Launchpad machinery will do its magic and add more debug information to the bug report [16:25] the next step would be trying to confirm the issue [16:26] if you can confirm it and maybe add some more details to what happened, you can have a look and see if it was already reports in the upstream bug tracker [16:26] if it was, see if you can add more information to the bug [16:26] if it wasn't report it [16:26] sometimes it's useful to have a look and see if upstreams have some kind of "debugging information" documented somewhere [16:27] for some of them it will be something like "start the program with --debug option" or something [16:27] a lot of that kind of information is listed at https://wiki.ubuntu.com/DebuggingProcedures and you should add info for your upstream project if you have any :) [16:28] also Launchpad has a great feature: you can link the upstream bug to the Ubuntu bug, right? [16:28] yeah [16:28] let me show you about that === godzilla is now known as Guest30844 [16:29] actually, give me a minute to find a bug [16:29] dholbach: do a dance! [16:29] * dholbach puts on some music and does the funky Upstream-Bug-Linking dance [16:29] https://wiki.ubuntu.com/Bugs/Upstream [16:30] ok so here is some documentation, depending on the bug [16:30] however, in general, you can just click "Also affects project" in launchpad, and paste in the bug url from the other bug tracker [16:30] https://wiki.ubuntu.com/Bugs/Watches [16:30] they looks like this [16:30] so you see there, on the top example, there's the bug in ubuntu, and the bug in the upstream project's bug tracker [16:30] in this case linux kernel bug 10911 [16:31] these are useful because with certain bug trackers [16:31] like mozilla's bugzilla, we can do neat things like syncing comments between the 2 bug trackers [16:32] https://bugs.edge.launchpad.net/ubuntu/+source/notify-osd/+bug/501393 [16:32] here is a good example of both ubuntu and mozilla developers working on a bug for example [16:32] however, just linking the bug can help [16:32] because it gives everyone a place to gather discussion around a bug [16:33] and especially for new users that might not know how things work [16:33] what other information can we help upstreams with dholbach? [16:33] tell me about this schedule [16:33] another fine thing you can do is: plan a Bug Day with your Upstream: invite them to https://wiki.ubuntu.com/UbuntuBugDay - triage bugs together, give people some coaching with that particular software - once you're good friends with your upstream, sky's the limit [16:34] alright... release schedule! [16:34] upstreams ask me all the time about the schedule [16:34] but sometimes I don't know what means what! [16:34] have a look at https://wiki.ubuntu.com/ReleaseSchedule [16:34] it shows our current release cycle [16:35] you can see how it goes from green in the beginning of the 6 months cycle to red at the end [16:35] green doesn't mean "it's all good, everything works" :-) [16:35] it rather means "everything's allowed from a developer point of view" [16:36] so we start with setting up the build tools in the beginning, plan features at UDS and merge with Debian (more on merges, syncs and Debian in a bit :-)) [16:37] at some stage we hit feature freeze, that's where we plan to have most of the feature in the distro [16:37] Xazax | why is the release cycle for upstream projects important? [16:37] doesn't mean they have to be all perfect [16:37] Xazax: will get to that in a sec [16:37] but it means that they have to be there and be test-able :) [16:38] so we move from feature development to fixing [16:38] and stabilisation [16:38] the later it gets in the release cycle the more conservative we get [16:38] to upload a small patch that fixes a crash 3 weeks before release is probably fine [16:38] but to upload a GIANT new release with n+1 new features will probably get you in trouble with the release managers [16:39] Jorge and I worked on a cheatsheet that makes it easier to understand what is allowed at which stage of the release cycle to ease the planning and coordination somewhat: http://people.canonical.com/~dholbach/cheatsheet.pdf [16:39] that's linked from https://wiki.ubuntu.com/Upstream/Adopt too [16:39] I hope that answers Xazax's question somewhat. [16:40] i would like to chime in on his question [16:40] sure [16:40] with things like a Long Term Support release it's critical for us as a downstream distro to communicate our release schedule [16:40] we sync our schedule with GNOME [16:40] so, we're kind of lined up [16:41] some upstreams want to know when they should release to have their software in ubuntu [16:41] so they can make a release, or decided to skip an ubuntu release, etc. [16:41] so, when we're at UDS [16:41] and we're talking to upstreams we make decisions on things dependent on release cycles [16:42] so it might be "well, that feature won't be ready in time, so we'll go with version 1.x instead" or something [16:42] that sounds very reasonable [16:42] whatever we can't fix during the release time, we have to fix after the release... jcastro: do you know how that works? [16:42] yes [16:43] it's a big source of confusion [16:43] so let's talk about Stable Release Updates [16:43] I think it all started with something like this: http://blog.somekool.net/images/img_4232.jpg [16:43] https://wiki.ubuntu.com/StableReleaseUpdates [16:43] doh! [16:43] shall I tell the story quickly before you move on? [16:43] yes! [16:43] ok [16:44] so in the very early days of Ubuntu we uploaded a couple of fixes (they all were upstream already) to the xserver to a stable release [16:44] that means that users who had installed the release (and not the development version) got those fixes [16:44] the developer had tested them all, they were upstream fixes and everything seemed good [16:45] the problem was that a certain percentage of our users (x% of some millions) suddenly had a broken Xserver [16:45] which resulted in people like my dad having to wrestle with: http://blog.somekool.net/images/img_4232.jpg [16:45] so we realised that we need to be more conservative with fixes like that [16:46] that's where SRUs originate from [16:46] jcastro: so how does this all work now? [16:46] https://wiki.ubuntu.com/StableReleaseUpdates [16:46] we have these [16:47] don't worry, it only looks scary! [16:47] the key point to remember here when talking to upstreams [16:47] is that really bad bugs should be communicated to distros [16:47] you see this all the time on like planet gnome [16:48] "wow, we found a bad bug in gstreamer, distros should ship this fix: $url" or something [16:48] as you hang out and talk with upstreams [16:48] at some point they will have a critical bug that will need to be fixed [16:48] so usually you can take ownership of that, talk to the ubuntu maintainer, and communicate the needs that way [16:48] sometimes upstreams fix small bugs [16:48] dhasenan: thanks [16:49] and of course those are good [16:49] whhooops [16:49] dholbach: thanks [16:49] * kfogel watches out for nick completion [16:49] but it can be frustrating to an upstream when the fix doesn't get into an SRU [16:49] it's just one of those things where the fix needs to be weighed with the risk [16:50] https://wiki.ubuntu.com/StableReleaseUpdates is pretty good at explaining what kind of fixes we want and which make sense [16:50] In general I think we do a good job here, but if an upstream still wants new crack in the distro, we have things like PPAs and backports [16:50] that's also an area where you can contribute [16:50] many ubuntu contributors set up and run PPAs as a service to upstreams for apps people love [16:50] http://people.canonical.com/~dholbach/cheatsheet.pdf should give you an overview over what to do in which case [16:51] that is highly encouraged, and a good way to learn packaging if you want to start the road to becoming an ubuntu developer [16:51] we only have 5 minutes left, so let's talk about debian for a bit [16:52] one of our most important upstreams is Debian [16:52] and we inherit a lot of packages and lots of great work from Debian developers [16:52] usually it works like this: if a package is unmodified in Ubuntu and we're before DebianImportFreeze (Check the release schedule again), it gets "synced" automatically [16:53] which means that the package in Ubuntu will get overwritten with the source that is in Debian right now (and rebuilt) === zen is now known as Guest90818 [16:53] if it's modified in Ubuntu, we need to figure out if we can drop out changes (then we'll ask for a sync) [16:53] and if we can't drop our changes, we need to merge the changes semi-manually [16:54] https://wiki.ubuntu.com/UbuntuDevelopment/Merging explains how the merging works and which tools we use [16:54] common scenarios for differences between Ubuntu and Debian are: [16:54] - we're in different parts of our release cycle and we decide to ship a newer version [16:55] - new improvements we want to try out in Ubuntu (launchpad integration of gnome apps, new indicators, new notifications, etc.) [16:55] and lots of other small things because of different things we support and not support [16:55] but generally the idea is: the more we can have in sync with Debian, the better [16:55] it means less merging, less maintenance hassle and a common code base to look at [16:56] so whatever is suitable in terms of patches to forward to Debian, we do forward [16:56] https://wiki.ubuntu.com/Debian/Bugs has more details on that [16:56] whew [16:56] that was alot of info! [16:56] and only one question, so either we were really great or really bad! [16:57] \o/ [16:57] that's the great thing about working on a certain piece of software is you get to know: lots of users (and make them happy with fixes), the upstream maintainers, lots of great Ubuntu developers that work with you and maintainers of others distros too [16:57] you not only make the world a better place by working on making software better for everybody, but you also get to know a lot of great people AND learn something new :) [16:58] one page you should definitely bookmark is: https://wiki.ubuntu.com/Upstream/Adopt :) [16:58] any final questions? [16:58] jcastro: any final words? :) [16:58] nope, thanks for coming everyone! [16:59] thanks everybody - you ROCK! }}}