Ubuntu Open Week - Make X rock! - Bryce Harrington - Thu, May 1, 2008
[19:01] <tonyyarusso> All right bryce, it looks like it's time to go. [19:02] <tonyyarusso> Bryce will be talking about the X Windows System, which provides the graphical environment in Ubuntu. [19:02] <bryce> hi all! [19:02] <bryce> The title of this talk was to be "Make X Kick Ass" but I guess that didn't pass the censors. ;-) [19:02] <bryce> X is involved in everything the user sees or touches [19:02] <bryce> Yet X is probably one of the most ignored parts of the desktop! [19:03] <bryce> You can make a *huge* impact on Ubuntu's quality by improving X in two ways: Bug squishing, and coding up features. [19:03] <bryce> In this session, we'll talk about how to contribute to these areas, and to help make Ubuntu's X kick more ass. [19:03] <bryce> The Ubuntu X team has a mailing list you can subscribe to (low traffic): [19:03] <bryce> http://lists.ubuntu.com/mailman/listinfo/Ubuntu-x [19:03] <bryce> Most of our work and discussions occur on IRC: [19:03] <bryce> FreeNode: #ubuntu-x [19:03] <bryce> You can join the Ubuntu-X bug team here: [19:03] <bryce> https://launchpad.net/~ubuntu-x-swat === tonyyarusso changed the topic of #ubuntu-classroom to: Ubuntu Open Week | Information and Logs: https://wiki.ubuntu.com/UbuntuOpenWeek | How to ask questions: https://wiki.ubuntu.com/UbuntuOpenWeek/Rules | Ask question in #ubuntu-classroom-cht, prefaced with "QUESTION:" | See https://wiki.ubuntu.com/UbuntuOpenWeek/JoiningIn to filter out channel noise | "Making X Rock!" - Bryce Harrrington [19:04] <bryce> For those who don't know me already, I'm Bryce Harrington and work on Xorg and related components. Timo Aaltonen, Tormod Volden, and unggnu are very active community members who do a huge amount of bug and package work. [19:04] <bryce> we're always looking to welcome new people to the team too! :-) [19:04] <bryce> First up is bugs. There's basically four ways to contribute to bug work: [19:04] <bryce> a) reporting - https://wiki.ubuntu.com/X/Reporting [19:05] <bryce> b) triaging - https://wiki.ubuntu.com/X/Triaging [19:05] <bryce> c) researching [19:05] <bryce> d) fixing [19:05] <bryce> All of these are important, but they require different levels of experience. [19:05] <bryce> I'm mostly going to focus on (c) and (d) here since the first two have been thoroughly covered in previous sessions, but first a few brief comments on reporting and triaging for X. === tonyyarusso changed the topic of #ubuntu-classroom to: Ubuntu Open Week | Information and Logs: https://wiki.ubuntu.com/UbuntuOpenWeek | How to ask questions: https://wiki.ubuntu.com/UbuntuOpenWeek/Rules | Ask question in #ubuntu-classroom-cht, prefaced with "QUESTION:" | See https://wiki.ubuntu.com/UbuntuOpenWeek/JoiningIn to filter out channel noise | "Making X Rock!" - Bryce Harrington [19:05] <bryce> For reporting X bugs the #1 most important thing to remember is to ALWAYS attach your Xorg.0.log. [19:06] <bryce> This file is thick with useful info like config settings, module versions, error messages, etc. [19:06] <bryce> The #2 thing to remember, is to PICK A GOOD TITLE. [19:06] <bryce> Too often people use a generic title like "X crashes randomly" or "Black screen after startup", and then everyone and their cousin thinks they have the same bug, when really they just happen to have similar symptoms. [19:06] <bryce> The result is much gnashing of teeth, people not getting their bugs fixed, cats and dogs living together.... [19:07] <bryce> A lot more info on making good bug reports is at https://wiki.ubuntu.com/X/Reporting [19:07] <bryce> Regarding bug triaging, I want to thank everyone who has helped in triaging X bugs. [19:07] <bryce> This is important work, and every hour that you put in saves a developer an hour that they can use to focus on *fixing* the dang bugs. [19:07] <bryce> Triaging is perhaps the easiest way to get involved in Ubuntu X; there's plenty of bugs, and you can start with little technical know how, and just learn as you go. Pedro gave an excellent talk on this earlier in the week. [19:08] <bryce> https://wiki.ubuntu.com/MeetingLogs/openweekhardy/TriageBugs1 [19:08] <bryce> I've outlined some triaging "projects" for X here: https://wiki.ubuntu.com/X/Projects [19:09] <bryce> Since there's so many bugs, I find it useful to set myself a goal, so I know when I've "finished". [19:09] <bryce> For Hardy one of my goals was to reduce the total bug count for -intel to <100 by the release. [19:09] <bryce> that was tougher than I expected, but we did make that goal in time :-) [19:09] <bryce> For Intrepid, I want to do the same for -ati, which has around 170 currently [19:10] <bryce> As a more aggressive goal I want to try reducing the total Xorg bug count from 1600 to 1000, but we'll see there - that'll be quite tough. [19:11] <bryce> I have some plots here to keep track of bug progress - http://people.ubuntu.com/~bryce/Plots/ [19:11] <bryce> look at the 6-month chart for Intel under X, for open bugs, and you can see how the bug count got reduced during Hardy development [19:11] <bryce> ok, maybe a few questions at this point? [19:12] <bryce> would someone mind pasting a couple? [19:12] <tonyyarusso> progfou> QUESTION: what about making X start and stop faster too? does Ubuntu collaborate with Fedora on this project? see http://fedoraproject.org/wiki/Features/OneSecondX [19:12] <bryce> yep, we keep close eyes on what other distros are doing and have been looking at that one in particular [19:13] <bryce> a community member dug into that and found a few patches which are low risk, that we might pull in for 8.04.1 [19:13] <tonyyarusso> bryce: (say next when done answering) [19:13] <bryce> I'll talk a bit more about opportunities for working more closely with upstreams and other distros - this is going to be largely driven by our Ubunt-X community size [19:13] <bryce> ok, next [19:13] <tonyyarusso> gkatsev> QUESTION: i have problems with full screen video, mostly in flash, does this have anything to do with X, if yes, any ideas how to fix? [19:14] <bryce> it may indeed be an X issue, but there are several layers in the stack that could be at fault with video issues [19:15] <bryce> that is actually an excellent question to take us into the next section of the talk, so let me head on into that [19:15] <bryce> the next step after triage - once you know you have an issue such as "video is failing, maybe flash related?" or whatever, is Research. [19:16] <bryce> we've got a LOT of bugs like this, where we know a bit of basics about the problem, but don't know how to fix them. [19:16] <bryce> 1600 Xorg bugs at last count, in fact [19:16] <bryce> But I've noticed that non-developers can play a huge role in making these bugs solvable, without writing a line of code. [19:16] <bryce> Here are several techniques I've noticed as effective. [19:16] <bryce> a) Look for patches in other distros [19:16] <bryce> b) Look for patches upstream in Xorg [19:16] <bryce> c) Look in appropriate mailing lists for discussions about the problem [19:16] <bryce> d) Test the latest upstream version [19:16] <bryce> e) For regressions, test older versions [19:16] <bryce> f) Get a backtrace [19:16] <bryce> g) Talk to an upstream person about the bug [19:17] <bryce> Providing pointers for anything found (even if you're not certain it's relevant), can help a lot. [19:17] <bryce> People can try out and verify the fix, and packagers can (usually) easily grab the fix and package it up. [19:17] <bryce> Testing older or newer versions of X or the driver requires pulling them via git and compiling them [19:17] <bryce> which sounds technically challenging so probably many don't try it [19:17] <bryce> but this can be extremely effective at locating a patch to fix the problem. [19:18] <bryce> (I have some plans to help folks with pre-building git versions of drivers, but need to do more development on that before I can push it. I did a prototype for -intel last month that proved quite effective.) [19:18] <bryce> (http://people.ubuntu.com/~bryce/bisect/) [19:18] <bryce> If it works with the newest upstream version, then there may be a patch in the new version we could backport. [19:18] <bryce> If you find it worked with older versions but does not work in the latest, then this may identify the patch that broke things, which we could then revert or fix. [19:19] <bryce> Backtraces are awesome. X crashes are some of the most critical kinds of bugs we see. [19:19] <bryce> Similar to crashes are lockups and freezes. [19:19] <bryce> In all these cases, getting a backtrace can be extremely illuminating in identifying where X stopped. [19:19] <bryce> I'll talk more about backtraces in the next segment. [19:19] <bryce> Directions on obtaining backtraces this are available at this page, so I won't bore you with a lot of details, since this gives a nice paint-by-numbers approach: https://wiki.ubuntu.com/X/Backtracing [19:19] <bryce> So next time you crash X, don't just reboot! Instead, grab a second computer and ssh into the broken one, attach gdb, and get a full backtrace. :-) [19:20] <bryce> Okay, let's take more questions, particularly about bug reporting, triage, troubleshooting....? [19:20] <bryce> tonyyarusso: fire away with the next one [19:20] <tonyyarusso> rodolfo> QUESTION: so far we've seen X and Intel's video cards not working 100%. There are rumors that this will be changed until 8.10 official release. Is this correct? (specifically i9xx series) - perhaps tips on tracking down the bugs and subscribing? [19:21] <bryce> right, we've been working quite closely with Intel this past release on squishing a lot of bugs [19:22] <bryce> I mentioned earlier about the reduction of -intel bugs from 180 to 100 - obviously this leaves a lot still to fix. But we plan to continue work on getting the bugs solved. [19:22] <bryce> Intel has been *great* at responding to bug reports forwarded upstream to bugs.freedesktop.org [19:23] <bryce> they have very specific expectations about the bugs forwarded there though, so this is why I emphasize the bug triaging procedure so much - collecting all the right info, researching, backtraces, testing current git, etc. all are necessary for upstream to proceed with finding a fix [19:24] <bryce> we've got good processes for -intel and good momentum, and I'm quite hopeful to see this result in 8.10 being a very very solid -intel release. And it'll be even better with your involvement. [19:24] <bryce> tonyyarusso: next please? [19:24] <tonyyarusso> qense> QUESTION: Does intel have an own public bug tracker? Or are their employees just reading other bug trackers? [19:24] <bryce> Intel's employees read the Xorg bug tracker at bugs.freedesktop.org [19:25] <bryce> they don't have their own outside that. [19:25] <bryce> next [19:25] <tonyyarusso> gruber> QUESTION: What should people who complain about resolution be told to do if X can't probe for their monitor type or frequency data--how should the new user supply it? [19:25] <bryce> I like that question :-) [19:25] <bryce> in fact, things have gotten a LOT better since Feisty, when resolution problems were mentioned in *every* Ubuntu review [19:26] <bryce> I've got some very detailed directions on solving resolution issues on the X troubleshooting page in the link mentioned above, but a few quick tips: [19:26] <bryce> first, I always look in /var/log/Xorg.0.log for the EDID section, and then look at what the monitor is reporting, what the card says it's capable of, and what the result is. About 50% of the time this gives a clue. [19:27] <bryce> many of the common causes for resolution issues in the past have been eliminated. One of the biggest remaining ones is monitor or vid-card breakage, which is treated via Quirks. I'll talk more about quirks later. [19:28] <bryce> another issue is when the monitor simply won't report edid at all, for whatever reason [19:28] <bryce> if we can't even identify the monitor, quirks aren't going to help unfortunately. [19:29] <bryce> there is a tool, displayconfig-gtk, which we no longer use as a stock config tool since it is applicable for only a small number of situations, however it has a good monitor database [19:29] <bryce> be careful in using it though, and back up your xorg.conf, as it will be (attempting) to modify it [19:29] <bryce> next [19:29] <tonyyarusso> rzr> QUESTION: does it sound possible to run a repository of upstream snapshot built ? (for testing) (an apt-repo, I assume, PPA or other) [19:30] <bryce> right, that's essentially what my goals are with the bisect page aboe [19:31] <bryce> I can't do it with PPAs unfortunately since they only let you keep the newest versions [19:31] <bryce> next [19:31] <tonyyarusso> rodolfo> QUESTION: For Linux newbies, updating the driver sounds like a painful process. Specially when it's related to X. Maybe because of each file requirements. So, is there a guide for first-timers to update it or something like that available (a self-explained doc or tutorial)? [19:31] <bryce> indeed there is [19:31] <bryce> standby [19:32] <bryce> here we go - https://wiki.ubuntu.com/XorgOnTheEdge [19:32] <bryce> we have some scripts to help automate building of .debs and such there [19:32] <bryce> and tormod provides some prebuilts [19:33] <bryce> I also hope to expand on this during Intrepid to make it easier, since this is such a great way to troubleshoot problems, but also know people like to avoid compiling stuff (yet compiling builds character!) [19:33] <bryce> ok, I'm going to continue on with the talk [19:34] <bryce> Okay, now for the hard-core stuff on smashing bugs. Well, "hard-core" is probably a bit of a stretch... What I'm going to describe are more at the "apprentice developer" level - you need to know C, but not a lot of depth into X itself. [19:34] <bryce> Since we just talked about backtraces, let me explain really briefly what a developer does with them. [19:34] <bryce> Most crashes are because of bad pointers. Pointers are basically just addresses to places in memory. [19:34] <bryce> Pointers can go bad in several different ways. [19:34] <bryce> The most common are NULL pointer dereferences. Basically, pointers are set to 0 (aka NULL) to show they're not valid. If you try to access something using a pointer with value 0, that's a serious error and causes the program to fault. [19:34] <bryce> You can usually spot NULL pointers right in the backtrace - some variable set to 0x0. [19:34] <bryce> Another common pointer error is using an uninitialized pointer, that weren't set to NULL and instead point to a random place in memory. These are harder to spot, and usually require studying the source code. [19:34] <bryce> If you know C and are comfortable with pointers, this can even be kind of fun. :-) [19:34] <bryce> (well, I find it fun, maybe I'm crazy) [19:35] <bryce> If you don't know C, don't worry about all the pointer jibberish. The important thing to remember is: If it crashes (or locks up, or freezes, or boots you back to the login screen) get a backtrace. [19:35] <bryce> [19:35] <bryce> Another apprentice-level X developer task are Quirks, like I mentioned earlier. [19:35] <bryce> Quirks are basically hardware-specific fix-ups, and can be specific to a monitor, or to a graphics card. [19:35] <bryce> When X starts up, it gathers the 'EDID' data from your monitor to get your monitor's id, and then looks up if there are quirks for that id to apply. X does the same with your graphics card, using your pci id. [19:35] <bryce> Monitor quirks are kept in the X server, in the file hw/xfree86/modes/xf86EdidModes.c. [19:35] <bryce> Here is a listing of the current quirks: [19:36] <bryce> quirk_prefer_large_60: Detailed timing is not preferred, use largest mode at 60Hz [19:36] <bryce> quirk_135_clock_too_high: Recommended 135MHz pixel clock is too high [19:36] <bryce> quirk_prefer_large_75: Detailed timing is not preferred, use largest mode at 75Hz [19:36] <bryce> quirk_detailed_h_in_cm: Detailed timings give horizontal size in cm. [19:36] <bryce> quirk_detailed_v_in_cm: Detailed timings give vertical size in cm. [19:36] <bryce> quirk_detailed_use_maximum_size: Detailed timings give sizes in cm. [19:36] <bryce> quirk_first_detailed_preferred: First detailed timing was not marked as preferred. [19:36] <bryce> quirk_detailed_sync_pp: Use +hsync +vsync for detailed timing. [19:36] <bryce> At first glance it's not obvious what these mean, but basically they deal with issues where the monitor manufacturer didn't encode EDID properly (like putting values in centimeters rather than millimeters, etc.) [19:36] <bryce> EDID == Extended Display Identification Data [19:37] <bryce> basically it's a magical chunk of binary data that is embedded in your monitor, that your computer can query plug-n-play-ishly [19:37] <bryce> If you're curious about the particulars of any of these quirks, if you look in the source of that file it references bug ID's that explain the problem in better detail. [19:38] <bryce> [19:38] <bryce> Graphics cards also have quirks. These are stored in the video driver. [19:38] <bryce> For example, with Intel graphics, this is in xserver-xorg-video-intel in the file src/i830_quirks. [19:38] <bryce> Here's the current list of available quirks: [19:38] <bryce> quirk_mac_mini [19:38] <bryce> quirk_ignore_tv -- the most common quirk [19:38] <bryce> quirk_lenovo_tv_dmi [19:38] <bryce> quirk_ivch_dvob [19:38] <bryce> quirk_pipea_force -- another common quirk [19:38] <bryce> Again, look for bug reports for explanations of these. [19:38] <bryce> Believe it or not, us Ubuntu-ers have been the source for most of upstream's X quirk data . [19:38] <bryce> This is because Ubuntu kicks ass! [19:39] <bryce> ... okay, really it's because there are more Ubuntu users than other distros, so we just have a more widespread coverage of desktop hardware. [19:39] <bryce> But quirks are a good way for us to contribute upstream, and so it's important we recognize bugs that just need quirks made for them. [19:39] <bryce> tonyyarusso: ok time for a few more questions... next? [19:39] <tonyyarusso> phoenix24> QUESTION: Would you give an overview of the X architecture or Design ? [19:39] <tonyyarusso> (briefly, I'm sure) [19:39] <bryce> hehe [19:40] <bryce> ok, well veerrry briefly, the whole X stack is broken into a client / server model [19:40] <bryce> the server is, obviously, xserver (although some people run xgl which is basically an alternate server) [19:41] <bryce> clients are basically anything that runs on X - gui apps, cmdline tools like xrandr or xdpyinfo, window managers, etc. etc. [19:42] <bryce> there are various protocols that these use to talk to one another. An example many have probably heard of recently is the XRandR protocol [19:42] <bryce> xlib (recently replaced by xcb) provides the interface between them [19:43] <bryce> the xserver itself is mainly a big chunk of code, with several libraries broken out, and a series of different drivers [19:43] <bryce> there are two kinds of drivers - input and output. Input are keyboard, mice, trackpads, etc. Output are basically video cards. [19:44] <bryce> by and large, most of the stuff we find "interesting" is either in the xserver or in one or more of the drivers [19:44] <bryce> hopefully that gives a sufficient overview [19:44] <bryce> tonyyarusso: next please? [19:44] <tonyyarusso> progfou> QUESTION: does the Ubuntu-X team has plan to make X.Org run as a non-root user to make bugs less critical on a security point of view? or do we have to wait for upstream to implement something missing first? [19:46] <bryce> progfou, upstream is working on making Xorg run as non-root; there are a variety of projects under way upstream like this [19:46] <bryce> the level of participation is driven largely by the amount of community participation in the Ubuntu X team, and obviously by their own personal itches they wish to scratch [19:47] <bryce> worst case, we can pull in stuff from upstream as its ready, but it would be preferable to have people involved in assisting with pulling the bits in to test earlier [19:47] <bryce> next [19:47] <tonyyarusso> mariusss> QUESTION: Hi Bryce! With the latest changes in X and the video drivers (nvidia in my case) everything started to be more automatical (configuration of the monitor, video card, etc). But, are you aware of the fact the some of the latest video cards (expensive ones such as [19:47] <tonyyarusso> GeForce 9800GTX) or some of te worst (most of the on-board ones) are a pain in the a** to make them work in Hardy (twith or without a video driver installed)? [19:47] <bryce> I joke with Kees that the easiest way to make brand new hardware work is to stick it in your closet for a few months. ;-) [19:48] <bryce> yes, it has always been true that the newest hardware tends to not be as well supported as older stuff [19:49] <bryce> we've been doing really well with Intel due to our level of interaction with them, and I hope with ATI's recent increase involvement with the open source community that we'll see big improvements there too (we already have a little) [19:50] <bryce> as a general rule, the best way for these is to get good bugs reported on them, and forward upstream to Xorg. [19:50] <bryce> ok, let me continue on with the talk [19:50] <bryce> We can divide feature coding into three categories: [19:50] <bryce> a) X config tools [19:50] <bryce> b) Packager tools and scripts [19:50] <bryce> c) Core X and driver coding [19:51] <bryce> X configuration tools are things like displayconfig-gtk or the new Screen Resolution tool, but could also include command line or install-time tools like xresprobe, bulletproof-x, xfix, etc. [19:51] <bryce> It is important that work on X config tools be done in a manner that upstream will accept, so we can contribute them and benefit from their assistance in maintaining them. [19:51] <bryce> GNOME and Xorg prefer tools be written in C, for example. [19:51] <bryce> [19:51] <bryce> One config tool we will need in the coming year is an GUI XInput Config tool. [19:51] <bryce> This would allow run-time configuration of mice, keyboards, tablets, and other input devices. [19:51] <bryce> To start, we need it to be conceptualized - what screens it should include, how they should be laid out, etc. [19:51] <bryce> Prototypes could be done in Python, but ultimately what we ship will need to be written in C. [19:51] <bryce> Let me know if this is something you'd like to work on. [19:51] <bryce> [19:51] <bryce> There are also a number of improvements that could be made to Screen Resolution, which I've listed along with some other projects here: [19:51] <bryce> https://wiki.ubuntu.com/X/Projects [19:51] <bryce> [19:51] <bryce> Finally, let's talk a bit about core X development work (the Advanced projects at the above url). [19:52] <bryce> This touches on the fast X boot question. [19:52] <bryce> Historically Ubuntu has contributed only lightly on core X - mostly bug fixes and fringe work, [19:52] <bryce> but I know there are some very smart Ubuntu-ers coming on board and looking for respectible challenges, and core X development certainly fits! [19:52] <bryce> [19:52] <bryce> You can read elsewhere about what is going on upstream that we could [19:52] <bryce> participate in. The recent XDC2008 notes are the best place to start: [19:52] <bryce> http://www.x.org/wiki/Events/XDC2008/Notes [19:52] <bryce> One of the Ubuntu-X team's top priorities for Intrepid is Input Hotplug. We'd love to see additional testers get involved in this. [19:52] <bryce> Some other hot topics include kernel modesetting, fast X boot, TTM memory management, redirected direct rendering for GL/Xv, and multi-screen infrastructure. [19:52] <bryce> These are all under way upstream or at RedHat, and will come to us eventually. [19:52] <bryce> But one way to accelerate when we see them on Ubuntu would be to try building and testing them on Ubuntu, identify and fix problems, and participate upstream in their development. [19:53] <bryce> [19:53] <bryce> Some other areas not under way upstream as far as I know, but that would be cool to see for Ubuntu include: [19:53] <bryce> Multi-pointer X (MPX) - for using tablet and mouse in conjunction. Useful for drawing apps and maybe games. [19:53] <bryce> OpenGL with KVM - need to evaluate and/or port VGML to Ubuntu [19:53] <bryce> OpenGL performance for gaming - would be great to help optimize 3D for OpenGL games, to make it more robust and smoother under Ubuntu. [19:53] <bryce> Phoronix published a perf test that would be interesting to run and examine on Ubuntu. [19:53] <bryce> [19:55] <bryce> I don't want to set false expectations that we have definitive plans for working on any of these, except for Input Hotplug, but to list them as areas of possible involvement [19:56] <bryce> Here again is where to subscribe to join the Ubuntu-X project: [19:56] <bryce> http://lists.ubuntu.com/mailman/listinfo/Ubuntu-x [19:56] <bryce> FreeNode: #ubuntu-x [19:56] <bryce> https://launchpad.net/~ubuntu-x-swat/ [19:56] <bryce> tonyyarusso: next questions? [19:56] <tonyyarusso> rzr> QUESTION: what are priority for ubuntu if you have to choose between hw support or reliability or performance ? (or features) [19:57] <bryce> for an LTS release, reliability is the highest priority, with hw support second and performance third [19:57] <bryce> for a non-LTS release, I'd probably set the three to roughly equal in importance, but with increased importance on reliability as the release approaches [19:57] <bryce> next [19:57] <jcastro> that was the last question [19:57] <bryce> excellent! [19:57] <jcastro> Thanks for filling us in on X Bryce! [19:57] <tonyyarusso> bryce: bumping up against the next one now. [19:58] <tonyyarusso> bryce: There were more questions though - should they join you in #ubuntu-x for those? [19:58] <bryce> ok, well again, please join us in #ubuntu-x if you'd like to know more, or join in [19:58] <bryce> yep [19:58] <tonyyarusso> sounds good [19:58] <tonyyarusso> Thanks bryce !