Troubleshooting

Revision 18 as of 2008-09-10 19:22:08

Clear message

For hard bugs the analysis phase is the most important, and most challenging part of bug work. Depending on how the bug is behaving, there are multiple directions to investigate the issue. Here's some different approaches:

Problem manifested only recently

If the issue has been narrowed to occur only after (or before) a given point in time or software version, then it is possible to narrow in on the specific cause of the issue through a "Bi-Section" strategy.

Essentially, if you know it occurred in Version 1, but not Version 8, have a person able to replicate the issue try Version 4. If it's there, then have them try Version 6, otherwise Version 2.

If the problem is in the current Ubuntu, but not in the prior Ubuntu, it can be useful to have them test the intermediate Alpha versions of the new release.

Once you have bracketed it down to a specific version of something, you can then go through the individual patches included in that version compared with the prior one. Sometimes the patch descriptions can give a strong clue to this. If there are a number of changes, then rather than trying each patch one-by-one you may want to simply disable the latter half of patches, and bisect that way.

If you've narrowed it to an upstream version change, then you may wish to use git's bisecting functionality to assist with this.

X starts up with the wrong video driver (e.g. vesa)

The current X server attempts to auto-configure itself, which works most of the time, but occasionally will result in an improperly selected video driver. In some cases it may not be able to determine the right driver at all, and will default to -vesa.

The lookup of pci id to driver name is done in videoPtrToDriverName() and chooseVideoDriver() in hw/xfree86/common/xf86AutoConfig.c. This uses pci ids listed in .ids files in /usr/share/xserver-xorg/pci/. These files are generated by the makefile of the appropriate video driver.

If you have a very unusual video driver (like a new one, or one supplied by a 3rd party vendor), you may need to patch videoPtrToDriverName() to add it.

Or, if you have a new-ish video card but are using a fairly standard driver, you may simply need to update the pci id list. Try editing the appropriate file in /usr/share/xserver-xorg/pci/ to add it; if that works, then simply send in a bug report requesting addition of that pci id to the driver's code, and a developer can take care of adding the appropriate pci id to the driver's source (for example, with -intel it goes in src/common.h, while -ati has it in src/ati_pciids_gen.h).

Problem manifests only with specific configuration options

Often, a bug exhibits itself only with a specific configuration setting turned on (or off). Often, knowing that the bug appears with one option but not another provides a very strong clue for further debugging; it can also provide a short-term workaround for people until it's fixed. Here's a few options that are worthwhile to vary:

Setting

Alternate(s)

Comment

Section "Module"

Load "dri"

Disable "dri"

Many bugs (esp. on -intel) exhibit only with direct rendering enabled

Load "glx"

Disable "glx"

Section "Monitor"

HorizSync 28.0 - 80.0
VertRefresh 48.0 - 75.0

Comment lines out

Xorg 1.3+ often can figure these out automatically (and usually better). Hardcoding these incorrectly can lead to resolution, dpi, and other mode issues

Section "Device"

Driver "..."

Install alternate driver

Option "DRI" "true"

Option "DRI" "false"

Option "VBERestore" "true"

Option "VBERestore" "false"

Option "XAANoOffscreenPixmaps" "true"

Option "XAANoOffscreenPixmaps" "false"

Section "ServerLayout"

Option "AIGLX" "true"
Or blank

Option "AIGLX" "false"

AIGLX is already enabled in the server so is on by default. Turning it off might affect performance or AIGLX-related issues.

Section "Extensions"

Option "Composite" "Enable"

Option "Composite" "Disable"

Composite is enabled by default starting with Ubuntu. It can cause issues on some systems in some conditions; this is one way to turn it off.

Problem manifests only with a particular driver

If the research found that most people with this problem were all using the same driver, then obviously it makes sense to explore it from that aspect.

Note that for most graphics hardware, there are at least two different drivers. It can be worthwhile to test the alternate driver to verify it's a driver issue.

  • NVidia: -nv (open) and -nvidia (proprietary)
  • ATI: -ati (open) and -fglrx (proprietary)
  • Intel: -intel (current) and -i810 (legacy).

Each driver has its own source code package, which can be retrieved via xserver-xorg-video-<driver>. The open source drivers also have git repositories at http://gitweb.freedesktop.org.

Resolving these issues will generally require patching the driver code, although some driver-specific issues end up requiring changes to other pieces of code, like the xserver.

The following can be added to your /etc/X11/xorg.conf to provide additional debug information:

Section "Device"
        ...
        Option "ModeDebug" "true"
        ...
EndSection

Problem manifests only with particular kind of hardware

Many issues are highly specific to a particular kind of hardware, such as only Intel 855, or only a particular monitor model. Sometimes these end up being general bugs, but often they require adding hardware-specific quirks to the driver or to xserver.

Xorg crashes often tend to be hardware specific, as do issues with TV out and resolution detection failures.

Because they are hardware specific, it's required to provide your hardware identifiers. For video cards, this is the PCI ID, which can be found via lspci -vvnn. For monitors, you should provide the exact model id and manufacturer; EDID information (from ddcprobe or via read-edid by get-edid | parse-edid) can also sometimes be useful.

Problem manifests under seemingly random conditions

Few bugs are truly random; usually this just means more data is needed. In any case, it's nearly impossible to "troubleshoot-by-proxy" Xorg bugs that occur randomly, so it's important that the reporter do some extra work to either a) reduce it to a reproducible case, or b) identify the location in the code where the fault occurs by following other recommendations in this document (e.g. if it is crashing, use the directions for obtaining a backtrace).

Here are some tips for turning a random issue into a deterministic one:

  • Does it seem to come on only after running the screensaver? Or perhaps after resuming from suspend, hibernate, or other power savings mode?
  • Examine resource utilization over time - could it be triggered by a high memory or cpu load?
  • When you experience it, write down everything you did over the last 5 minutes or so; then, the next time it occurs, look at the list and see if you were doing some of those same things. Repeat those actions and see if you can trigger the problem.
  • If you boot a newer or older Ubuntu LiveCD on the same hardware, do you see the same issue?
  • If there is an alternate video driver available for your hardware, can you recreate the problem with that driver?
  • Could it be caused by fluctuating Network/power conditions?

Problem manifests itself during video playback

Many video bugs have already been reported, are well-known, and require significant architectural work to be done upstream in order to resolve them. So, before reporting a bug, please review existing bug reports to see if the issue is already known. Many of the known issues relate to use of Xv and/or compiz, so typical workarounds include not using one or the other or both.

If you are finding your issue is not already reported, first identify which Xorg extensions are involved. You can isolate the issue by varying some of these (e.g. trying with or without Compiz, comparing with XAA vs. EXA, enable/disable DRI, etc.) and by testing out different options in your video player (like whether to use Xv).

Problem manifests itself when using 3D software (compiz, games, GL...)

Many of the open source Xorg drivers do better with 2D than 3D, so issues are not uncommon. Unfortunately, this also means that most "solutions" will involve switching to one of the proprietary binary drivers, or to open but experimental 3D drivers.

In general, these issues need to be forwarded upstream. Gather as much supporting evidence in the form of screenshots/screencasts, log files, lspci -vvnn and detailed steps to reproduce the issue.

If you are seeing the issue with a proprietary application such as a commercial game, first try to reproduce the issue with various open source games or tools such as glxgears; your bug will be much more likely to gain attention upstream if it can be demonstrated with software that upstream developers can easily get their hands on.

Problem manifests as a performance degradation issue

For general performance degradation, you can often isolate these issues by experimenting with different driver options, such as switching between XAA and EXA, turning DRI on or off, or using an alternate driver. Many of these issues will already be well-known, so be sure to search the bug tracker before reporting. They're also often issues that are not trivially solved, so you may need to either live with the issue or use a workaround (such as no Compiz) for a while.

For performance regressions in specific situations, first try to identify exactly when the issue started happening; try reverting to older versions of your video driver (perhaps by testing different alpha releases of Ubuntu). If you can narrow down the range of driver versions where the issue occurred, this can help a great deal in identifying the patch that caused the regression.

If the issue has been present for a long while, and is specific to a particular application, consider reporting the issue against that application. Even if it is a driver limitation at core, sometimes the application developers will be able to work around it on their end more easily than it can be handled in the driver.

If the performance issue occurs with 3D apps and/or with Compiz enabled, but not with 2D, this means your system's 3d acceleration (DRI) is not working properly. 3D can be accelerated either in hardware or software; if you expect hardware acceleration but your driver is falling back to software, that could be the problem. There have been some cases where EXA attempted to do partial hardware acceleration - which is worse than if it just did it all in software; in this case forcing software rendering will help the performance problem (though obviously it won't give the best performance your hardware can do). See http://cworth.org/talks/lca_2008/ for a thorough discussion.

During Hardy development, we ran into some severe performance problems with EXA on -intel, which resolved by setting 'Option "MigrationHeuristic" "greedy"' on by default, which seems to work well for everyone, but experimenting with other MigrationHeuristic values may be of use if you see performance loss with EXA but not with XAA. Option "ExaNoComposite" "true"/"false" is another option to play with.

Problem results in screen display corruption

Nearly all screen corruption issues will be due to a bug in a driver. Identify the driver and the specific steps to produce the corruption. Then run the xserver through gdb to identify the line or lines immediately prior to the corruption.

From here, things to try could include checking for invalid/undefined values, adding usleep() calls to add delay, or even disabling the lines in question.

Once a preliminary patch exists, post it to the upstream xorg list for feedback. Often they can suggest a better patch.

Problem results in X crash, lockup, freeze, or exit

In some cases, an error message will be printed before the fault; these can be used to identify where in the codebase the fault occurred, and often give an explanation as to why.

Otherwise, use gdb to get a backtrace. Once the issue is found, step through the code leading up to the line where the fault occurred. Look for invalid/undefined values, or questionable logic. Try disabling the line or lines where the fault occurred, adding usleep() before it, or etc.

Once a preliminary patch exists, post it to the upstream xorg list for feedback. Often they can suggest a better patch.

Problem involves wrong resolutions, refresh rates, or monitor specs

These sorts of issues used to be quite common in Feisty and earlier versions of Ubuntu, but should be quite rare these days. However, they do happen from time to time. They come in two sub-classes: Autodetection at installation, and available resolution options in tools.

In the first case, where X has successfully booted, but did so at an unexpected resolution (either too high, too low, the wrong aspect ratio, wrong refresh rate, etc.) the issue is likely an autodetection failure in xorg-server or in your video driver. Typically this only happens for hardware that is very new, very old, or very obscure; mainstream stuff ought to always get autodetected correctly.

In the second case, you may expect a range of different resolutions to be listed in the Screen Resolution tool, yet you only see a subset (or some other different set). Or potentially you might see a completely incorrect monitor shown. In these cases, the problem is likely to be that your monitor's EDID information was not correctly detected by the code in gnome-desktop (which is called from gnome-control-center and gnome-settings-daemon).

In either case, the troubleshooting process is similar.

1. Run xrandr and compare it's output with what you expected.

A common thing to check is if the TV Out output is enabled - this has been a problem especially on Intel hardware using the -intel driver; for this particular bug, the solution is to add a tv-out quirk to xserver-xorg-video-intel: Simply file a bug with the xrandr output and your chipset subsystem pci id (just include your Xorg.0.log and/or lspci -vvnn and we can get the pci id from that).

If xrandr shows the right resolutions, but the Screen Resolution tool isn't presenting that exact list to you, then that tool has a bug. File a bug including both the xrandr output and a screenshot of what Screen Resolution is offering (and of course always include your Xorg.0.log when reporting X bugs.) Meanwhile, you can use the xrandr command line tool to set your resolution.

If xrandr shows the right resolutions, but X just didn't pick the preferred one, there is probably some secondary thing going on. For instance, you may have stray gconf or .gnome2/monitors.xml settings - which you can check by logging in as a different user and seeing if you get the correct resolution. Or something else in the X startup processes may be interfering. Or perhaps your monitor failed to correctly indicate the 'preferred' resolution, or it did, but something in the process ignored that; this can be the case if you're using a CRT and want to use a resolution less than the maximum that your CRT and video card can support. Carry forth to #2.

If xrandr is *not* showing the right resolutions you expect, then there is something wrong at a lower level. Proceed on to #3.

2. Review your /var/log/Xorg.0.log.

While complex, reading through this file can show a lot of the decision making process that X goes through to decide what resolutions to use. It will indicate, for instance, resolutions that can't be used due to memory limitations, resolutions it thinks are out of range, and so on.

A particular issue to watch out for is if it has mis-understood what monitor you have attached, it may start throwing out perfectly good resolutions because it thinks they're out of sync range.

Other issues can occur if the EDID information for your monitor is incorrect; sometimes this can be fixed via quirks to xorg-server; other times you'll have to manually configure. Move on to #3.

3. Run sudo ddcprobe and/or sudo get-edid|parse-edid and see if the timings match with xrandr

A particular bug to look for is the results indicate an 'EDID fail'. If this happens, you'll probably have to manually configure your monitor settings in xorg.conf. See https://bugs.launchpad.net/ubuntu/+bug/edidfail

If there is a huge discrepancy between ddcprobe or get-edid, and what's available in xrandr, then something is borked up either in xorg-server or your video driver. File a bug with all the information collected so far.

4. Test working around manually

In general, you can bypass all the autodetection madness by using the old school method of configuring your hardware. In your xorg.conf, specify the correct device driver, monitor vrefresh and hsync, resolutions, bit depths, etc. If you can get it to work yourself, then file a bug with your original and fixed xorg.confs, along with your Xorg.0.log, the exact make and model of your monitor, the output from ddcprobe and get-edid, and so on. Often the situation is that a special quirk needs to be constructed for your hardware, and we'll need to work with upstream to determine it, so all this info will help fill in details.

If you are certain you have a correct configuration, and it *still* won't work, then it is possible you may need to generate your own modeline for the hardware. This is extraordinarily rare, but it can happen for really old or obscure monitors, or for situations where there is a very bad bug in the X code. Guides for creating modelines are googleable; please note there is risk of monitor damage if you issue a bad modeline, so take care to research properly before taking this route.

Problem involves wrong font dpi or size

In Feisty and before, GNOME hardcoded the dpi to 96. Starting with Gutsy, they dropped that and began trusting what the X server reported. However, in Gutsy the X server was not very trustworthy on this! Due to the way X was configured back then, using a set of ancient postinst heuristics, the screen physical dimensions were often wild guesses that rarely matched reality. So we saw a LOT of issues with huge or tiny fonts in the login screen, menu bars, applications, etc. For Gutsy, we ended up going back to hardcoding to 96 for the most part.

In Hardy, things have gotten a lot better. The hacky postinst script is no longer in use, so in many cases the X server's calculations are now correct. However, there are still many corner cases where the X server still doesn't get it right. So in the spirit of stability we stuck with the 96 dpi hardcoding that proved to work acceptably in Gutsy.

Unfortunately, this leaves us with various cases where the font dpi is not right. Luckily the issues are usually not the severe HUGE font sizes we used to see, but small things, like text not fitting properly in dialog windows, being slightly distorted on login screens, etc. You can check xdpyinfo and see what the X server thinks is the proper DPI, and if it looks ok to you, you can disable the hardcoded 96 dpi [TBD - how do override it?]

Problem involves buggy EDID from monitor

If the monitor is clearly advertising an incorrect mode (such as not advertising a preferred mode), a quirk can be added to the xserver to prefer a specific mode.

If you run either ddcprobe or get-edid | parse-edid (from the read-edid package), and see the error message "edid fail" in the output, then you have this issue.

For more background, troubleshooting approaches, and solutions, please see bug #194760.

Problem occurs on resume, logout, screensaver, power savings mode, tty switch, etc.

A large class of graphics bugs occur when changing modes, such as:

  • On resume from hibernate
  • On resume from suspend
  • On closing (or opening) laptop lid (independently of hibernate/suspend)
  • When screensaver comes on
  • When a GL screensaver comes on
  • Right after typing in password on the login screen
  • When switching to a tty console (e.g. ctrl-alt-f1)

Some reporters see issues on only one of the above situations, but many see it on a combination. If you experience one kind of crash from the above list, try some of the others to identify if you're seeing a single-mode or multiple-mode issue.

As well, the symptoms and characteristics of the failure can vary:

  • Screen is blank (like it's turned off)
  • Screen is black but being drawn (like it's on, but on a black screen)
  • Mouse cursor is visible and can be moved, but clicking does nothing
  • Mouse cursor is visible but won't move
  • Despite the display issue, the system works fine (startup sounds play, can ssh in, etc.)
  • Xorg crashes, returning user to the login screen
  • System seems locked up - can be pinged, but cannot ssh in nor run any programs
  • Xorg seems frozen - Ctrl-alt-backspace does not work
  • System seems frozen - Caps lock does not work
  • Occurs every time the mode changes
  • Occurs every other time the mode changes
  • Occurs only some times when the mode changes
  • Occurs only when there is a mode change after the system has been idle some period of time

Typically the failure will involve some combination of the above symptoms. When two reporters have the exact same set of symptoms for a given set of mode change failures, and are using the same chipset family, it's a good bet that they're experiencing the same bug. If the symptoms don't match up exactly, and they have the same hardware, then it's likely they aren't having quite the same bug, and the fix for one will probably not fix the other. On the other hand, sometimes a bug will exhibit different symptoms on different kinds of hardware. So on i915 it may show up as a system freeze, whereas on i945 it just crashes X, but a single bug fix will solve both issues.

Analysis and Workarounds

There are a number of tricks for working around these issues. In some cases they're good enough to address the user's needs. In other cases it simply changes the symptoms, or decreases the frequency of the issue. Sometimes they make things worse! But regardless, this analysis can provide very useful evidence to know which workarounds affect the issue, as that gives strong hints as to where the bug lays.

  • Disable Compiz
  • Disable all screen savers and power management; do some or all of the following:
    • Disable/remove screensaver. To remove:
      • rpm -ev gnome-screensaver
      • rpm -ev gnome-power-manager
    • Disable power management
    • Verify apmd is not running
    • xset s off
    • setterm -blank 0 -powersave off -powerdown 0
    • Check BIOS for any power management options
    • Configure X.org to turn off power management

Section "ServerFlags"
 Option "BlankTime" "0"
 Option "StandbyTime" "0"
 Option "SuspendTime" "0"
 Option "OffTime" "0"
EndSection
  • Force a different mode switch
    • Log out of X, and log into the console, and try suspend/resume (or hibernate, or whatever) from there
      • Running /etc/acpi/hibernate.sh or /etc/acpi/sleep.sh should do it
      • If it does not come up ok, the failure is in the kernel rather than X
      • Test if caps lock works; whether this works determines how bad the kernel failure is
    • Switch virtual consoles and back (alt-ctrl-f1, alt-ctrl-f7 or f8 or f9)
    • If laptop, close lid and reopen
    • Attach another external monitor
  • Try installing an alternate driver (like -i810 instead of -intel), or an older or newer version of the driver
  • Adjust ACPI settings
    • In /etc/default/acpi-support, set SAVE_VBE_STATE=false
    • In /etc/default/acpi-support, set SAVE_VIDEO_PCI_STATE=true
    • In /etc/default/acpi-support, set POST_VIDEO=false
    • In /etc/default/acpi-support, set USE_DPMS=true
    • Add the VBERestore option in Devices section of /etc/X11/xorg.conf
    • Remove the VBERestore option in Devices section of /etc/X11/xorg.conf
  • Adjust kernel ACPI settings in /boot/grub/menu.lst
    • Add to grub kernel parameters: "acpi_sleep=s3_bios,s3_mode"
    • Add to grub kernel parameters: "notsc"
  • Remove ACPI components to narrow down which causes the failure
    • sudo rmmod video

    • Check the contents of cat /proc/acpi/video/*/DOS before and after removing video
    • This can help identify if the bug is in the driver / BIOS
  • BIOS settings
    • Change "Internal Graphics Mode Select" BIOS setting from 1MB to 8MB

Problem involves keyboard input ceasing to work

Keyboard failures can occur for a large variety of reasons. Often, the nature of the failure can provide a strong clue as to which chunk of code actually failed. So start by testing the following, and mention them in your report:

  • Are you still able to move the mouse?
    • If so, can you interact with the window system? If so, then the xserver hasn't failed, so it's likely to be a keyboard input driver issue
  • Did the keyboard work for a while after booting into X, and then fail? If it didn't work at all, it may be a hardware/driver incompatibility (or maybe an irq conflict) - /var/log/Xorg.0.log, dmesg, and /var/log/syslog will be worth in depth review for keyboard-related error messages.
  • Try Alt-tab or tab to see if the window system responds. If so, then it may be an application or window manager level issue.
  • Does the keyboard "come back" after some time, or after hitting some keys like ctrl repeatedly?
  • Do all the keys fail, or only some of them?
    • Modifier keys (shift, alt, ctrl)?
    • Do numlock, capslock, and scroll cause the keyboard leds to light? If not, then it's probably a kernel issue
  • Try switching to a virtual terminal window (ctrl-alt-f1)
    • Does keyboard input work here? If so, then it is almost certainly an X-specific issue.
    • If not, try using the Magic SysRq key

      • alt-sysrq-h to see if the kernel is responding to magic sysrq commands
      • Try alt-sysrq-r to switch keyboard from raw mode to XLATE mode
    • Can you switch back to X (ctrl-alt-f7 or f9 typically)?
    • When switching back, does input work again?
  • Try exiting X using ctrl-alt-backspace. If this doesn't work, then the server has likely hung - see directions for troubleshooting X crashes.
  • Do you use a non-US keyboard layout? If so, can you recreate the issue when using a different layout? If not, it may be a keyboard layout specific issue.

Problem involves missing support for some keyboard keys

Many laptops (and some keyboards) come with non-standard keys that are found to be non-functional in Ubuntu, and need to be set up.

Here is the birds-eye view of how key mapping works: The kernel recognizes the key and establishes a raw scancode and maps it to an arbitrary kernel keycode (e.g. e06d -> 120). Xorg reads the kernel's keycode table, and generates it's own list of X keycodes (e.g. 120 -> 136). X then maps each X keycode to a keysym (e.g. 136 -> XF86AudioMedia). Finally, the window manager maps the keysym to a desired window manager shortcut action or an application command (e.g. XF86AudioMedia -> rhythmbox).

There are documented methods for setting up key changes locally (see 'More Info' below), but here we'll focus on how to fix the keys in the distro itself.

Part A: Kernel keyboard mapping

1. Find the "raw scancode" of the key in question. Typically for PS/2 keyboards this can be done from a tty console by hitting the key, and looking at dmesg output. For instance, if we are hitting the 'Media' key on a Dell Inspiron, we might see:

        atkbd.c: Unknown key pressed (translated set 2, code ... on isa0060/serio0).
        atkbd.c: Use 'setkeycodes e06d <keycode>' to make it known.

In the above, e06d is what we're looking for.

Another way to get the kernel keycodes is to shutdown X and hotkey-setup (e.g. /etc/init.d/gdm stop ; /etc/init.d/hotkey-setup stop) and then run showkey -s for raw scancodes or showkey -k for keycodes. Then restart hotkey-setup and gdm. This will give output for Media on a Dell Inspiron 1505N like:

0x9c
0x00 0x80 0xe2
0x80 0x81 0xe2

For USB controlled keys, an alternate approach is to run lshal -m from a console window and then hit the key. This should return something like:

    17:51:47.996: platform_i8042_i8042_KBD_port_logicaldev_input condition ButtonPressed = media

In this case, media is the key we're interested in. If ButtonPressed shows up as undefined, then the USB driver needs to be updated to support mapping this key to a Linux keycode (currently these are all compiled into the USB driver).

In some cases, your keys may be generating ACPI events instead of standard keycodes. To check this, run sudo tail -f /var/log/acpid and then hit the key. You should see something like:

received event "ibm/hotkey HKEY 00000080 00001010"

The interesting bits here is the part between the quotes. [What do we do with this info?]

On IBM/Lenovo laptops, some keys are only accessible via /dev/nvram. hotkey-setup can handle these and feeds the key presses into /dev/input/uinput to simulate regular key presses.

2. Identify the kernel keycode to use by referring to /usr/share/hotkey-setup/key-constants. For example, browsing through the file we find KEY_MEDIA:

...
KEY_BRIGHTNESSDOWN=224
KEY_BRIGHTNESSUP=225
KEY_MEDIA=226
KEY_SWITCHVIDEOMODE=227
KEY_KBDILLUMTOGGLE=228
...

These codes are defined in /usr/include/linux/input.h.

3. Set up the kernel keycode mapping for the keyboard layout by finding the manufacturer's file in /usr/share/hotkey-setup (part of the hotkey-setup package):

# inspiron multimedia keys
setkeycodes     e001    $KEY_PLAYPAUSE  # Dell E Key    Play/Pause (e001)
setkeycodes     e002    $KEY_STOP       # Dell i Key    Stop (e002)
setkeycodes     e003    $KEY_PREVIOUSSONG # Dell 1 Key  Previous Song (e003)
setkeycodes     e004    $KEY_NEXTSONG   # Dell 2 Key    Next Song (e004)
setkeycodes     e022    $KEY_PLAYPAUSE  # front         Play/Pause (e022)
setkeycodes     e010    $KEY_PREVIOUSSONG # panel       Previous Song (e010)
setkeycodes     e019    $KEY_NEXTSONG   # media         Next Song (e019)
setkeycodes     e024    $KEY_STOPCD     # buttons       Stop (e024)
setkeycodes     e06d    $KEY_MEDIA      # (Inspiron)    Media (e06d)

Here this adds a mapping for e06d to $KEY_MEDIA (kernel keycode 226).

These can also be set up via HAL quirks. See Keymap Quirk Debugger for details. Essentially, you need to add or update a section in the appropriate FDI file in /usr/share/hal/fdi/information/10freedesktop/, such as 30-keymap-dell.fdi for a Dell Inspiron 1505N:

...
        <!-- Dell Inspiron 9400, 640m, E1505/6400 -->
        <match key="/org/freedesktop/Hal/devices/computer:system.hardware.product" prefix_outof="MP061;MXC061;MM061"> 
          <append key="input.keymap.data" type="strlist">e005:brightnessdown</append> <!-- Fn+Down arrow Brightness Down -->
          <append key="input.keymap.data" type="strlist">e006:brightnessup</append> <!-- Fn+Up arrow Brightness Up -->
          <append key="input.keymap.data" type="strlist">e007:battery</append> <!-- Fn+F3 battery icon -->
          <append key="input.keymap.data" type="strlist">e008:wlan</append> <!-- Fn+F2 Toggle WiFi -->
          <append key="input.keymap.data" type="strlist">e008:bluetooth</append> <!-- FIXME Bluetooth switch (same as wireless) -->
          <append key="input.keymap.data" type="strlist">e009:ejectclosecd</append> <!-- Fn+F10 Eject CD -->
          <append key="input.keymap.data" type="strlist">e00a:suspend</append> <!-- Fn+F1 hibernate -->
          <append key="input.keymap.data" type="strlist">e00b:switchvideomode</append> <!-- Fn+F8 CRT/LCD -->
          <append key="input.keymap.data" type="strlist">e012:media</append> <!-- MediaDirect button (house icon) -->
          <append key="info.capabilities" type="strlist">input.keymap</append>
        </match>
...

So for instance, to support the e06d code we found before, we'd create a new section like the above, substituting in e06d for e012 (and any other codes that need remapping), into a new product section.

Use lshal | grep system.hardware.product and lshal | grep system.hardware.vendor to determine the product name and vendor of your laptop.

To see if hal has picked up your keymaps, restart the hal daemon and use lshal | grep keymap, which gives output like:

  info.callouts.add = {'hal-setup-keymap'} (string list)
  info.capabilities = {'input', 'input.keyboard', 'input.keypad', 'input.keys', 'input.keymap', 'button'} (string list)
  input.keymap.data = {'e005:brightnessdown', 'e006:brightnessup', 'e007:battery', 'e008:wlan', 'e008:bluetooth', 'e009:ejectclosecd', 'e00a:suspend', 'e00b:switchvideomode', 'e012:media'} (string list)

4. If there isn't a hotkey-setup file for your keyboard's manufacturer, you'll need to add a new one, and also add a reference to that file (and any associated logic) in the /etc/init.d/hotkey-setup script.

Part B: Xorg keyboard mapping

1. Xorg internally maps the kernel's keycodes into X keycodes, which apparently have nothing to do with kernel keycodes. To figure out what key was picked, run from a terminal window:

    xev

and hit the key in question. You should see output like this:

    KeyRelease event, serial 28, synthetic NO, window 0x3200001,
        root 0xb7, subw 0x0, time 137355697, (401,146), root:(413,264),
        state 0x10, '''keycode 136''' (keysym 0x0, NoSymbol), same_screen YES,
        XLookupString gives 0 bytes:

Note the keycode it specifies (136 in this example). Here's a way to filter the output more clearly: xev | sed -n 's/^.*keycode *\([0-9]\+\) *\(.*\), .*$/keycode \1 = \2/p'

Alternately, you can look up the scancode directly via /usr/share/X11/xkb/keycodes/xfree86. For instance, if the raw scancode is 'e06d', this corresponds to <I6D>:

    ...
    <I6D> =   237;
    ...

[To be honest, the above is a big guess at this mapping - can anyone confirm/deny the correctness of this?]

2. Now select a "keysym" to map. A listing of all available keysyms is in /usr/share/X11/XKeysymDB (or /usr/lib/X11/XKeysymDB before Edgy). Test the mapping:

    xmodmap -e "keycode <I6D> = XF86AudioMedia"

3. Next update or create a layout for the keyboard, by the appropriate file in /etc/X11/xkb/symbols/ (for media/internet keys, typically /etc/X11/xkb/symbols/inet:

// Laptop/notebook Dell Inspiron 8xxx
partial alphanumeric_keys
xkb_symbols "inspiron" {
    include "inet(media_common)"
    key <I02>   {       [ XF86AudioStop         ]       };
    key <I04>   {       [ XF86AudioNext         ]       };
    key <I21>   {       [ XF86Eject             ]       };
    key <I56>   {       [ XF86Display           ]       };
    key <I6D>   {       [ XF86AudioMedia        ]       };
    key <NFER>  {       [ XF86AudioPrev         ]       };
    key <XFER>  {       [ XF86AudioPlay, XF86AudioPause ] };
};

Then add the appropriate reference to /etc/X11/xkb/rules/xorg if it's not there already as is the case with inspiron, in the $inetkbds variable:

! $inetkbds = a4techKB21 a4techKBS8 a4_rfkb23 \
              acer_c300 acer_ferrari4k acer_laptop \
              acpi airkey \
              ...
              hpi6 hpxe3gc hpxe3gf hpxe4xxx hpxt1000 hpzt11xx \
              ipaq inspiron intel latitude \
              logiaccess logicd logicda logicink \
              ...
              toshiba_s3000 trust trustda \
              yahoo

and add a human description to /etc/X11/xkb/rules/xorg.lst:

! model
  pc101           Generic 101-key PC
  pc102           Generic 102-key (Intl) PC
  ...
  inspiron        Dell Laptop/notebook Inspiron 6xxx/8xxx
  precision_m     Dell Laptop/notebook Precision M series
  ...

and finally the localizable description to /etc/X11/xkb/rules/xorg.xml:

    <model>
      <configItem>
        <name>inspiron</name>
        <description>Dell Laptop/notebook Inspiron 6xxx/8xxx</description>
        <vendor>Dell</vendor>
      </configItem>
    </model>

After this, restart X, and the new keyboard layout can be selected in the usual manner.

Part C: Gnome/KDE hotkey mapping

In GNOME, the commands are set in gconf, via the Applications/System Tools/Configuration/global_keybindings and keybinding_commands keys. This is done in the file /var/lib/gconf/defaults/%gconf-tree.xml

More Info

Problem involves missing support for mouse, touchpad, or gamepad functions

Starting with Intrepid, Ubuntu now uses input-hotplug for most input devices. This section only addresses input-hotplug issues; if you're using an older version of the distro, please look elsewhere for troubleshooting directions.

For most devices you will need both a kernel driver and an X driver, in order for X to interpret its events correctly.

Kernel Driver

Use dmesg and lsmod to determine if the device is detected and the kernel module loaded. If not, you can load it using modprobe.

To determine the current kernel support level for your device, and to identify the kernel driver to use, here are some sites which may be of use:

* [http://www.qbik.ch/usb/devices/showdev.php?id=4410 Linux USB devices] * [http://kmuto.jp/debian/hcl/ Debian device driver check page] * [http://hardware4linux.info/ hardware4linux] * [http://www.ubuntuhcl.org/ Comprehensive Ubuntu Hardware Database]

If a kernel driver is not available in Ubuntu for your device, but one does exist somewhere, please file a wishlist bug against the "linux" product requesting it be added.

If a kernel driver is available in Ubuntu, but didn't get loaded automatically when you plugged in your device, file a bug against the "linux" product about this.

X Driver

For most devices, the evdev or evtouch drivers should be sufficient. These are used by default by X if no driver is listed in your xorg.conf, so if you have a different driver listed, try removing it or hardcode it to evdev.

In some cases, your device may require an X input driver other than evdev. If this is the case (e.g., it works when you hardcode the device in your xorg.conf), but it doesn't work automatically, the next step is to make sure the driver is specified in your HAL configuration.

X Input Device Configuration

If the right X driver is being selected for your device, but some of the settings are not quite right, you can adjust the settings using the HAL fdi files.

For more information on configuring HAL fdi files for X input devices, please see X/Config

Problem involves GUI application that crashes with an X error message

Some GUI applications print out an X error message when they crash, which can make it seem like the issue is in Xorg. In fact, while the error may indeed be generated by X code, most of the time it's due to the application making the call incorrectly or under invalid conditions. So these bugs need to be reported against the application, not X.

For GNOME applications, the X error can be caught using the following approach:

   gdk_error_trap_push ();

   /* ... Call the X function which may cause an error here ... */

   /* Flush the X queue to catch errors now. */
   gdk_flush ();
   if (gdk_error_trap_pop ())
     {
       /* ... Handle the error here ... */
     }

To determine the X error, it is useful to have a backtrace that includes both Xorg and GNOME debug symbols. See DebuggingProgramCrash for info on adding ddeb sources to your config. Then install libgtk2.0-0-dbgsym, and other gnome packages with -dbgsym appended.

Problem may be due to prior installs of a binary driver

Try:

 dpkg -l '*fglrx*'

and

 locate fglrx

to see if there is still some proprietary bits around causing problems.

Problem may be caused by gdm/kdm

If you suspect the issue may be caused by or in gdm, or for whatever reason you wish to isolate X from gdm, you can disable gdm from running at startup via:

  mv /etc/rc2.d/S30gdm /etc/rc2.d/disabled-S30gdm
  mv /etc/rc2.d/K30gdm /etc/rc2.d/disabled-K30gdm

Then start up X manually via:

  startx

You can subsequently restore gdm via:

  mv /etc/rc2.d/disabled-S30gdm /etc/rc2.d/S30gdm
  mv /etc/rc2.d/disabled-K30gdm /etc/rc2.d/K30gdm

Depending on how severe your login issue is, you may need to log into single user mode first to run the above commands.

A typical issue where you might want to do this, is if X is constantly crashing on bootup before you can log in, or if you can get to the login screen but can't access the tty consoles for whatever reason, but X is crashing when you login. This kind of situation is extremely rare, but unfortunately can happen with the right combination of bugs.

Note: Some docs you can find via google advocate running update-rc.d -f gdm remove, but this is incorrect - update-rc.d is a packaging utility, and not designed to be run by a user. It's changes will be undone automatically on the next package upgrade, so even though it may *seem* to work, it's accidental.

In the future, we hope to see gdm's startup manageable via upstart, which will provide a much cleaner and easier interface, but obviously such a change will take ample testing to make sure it's not introducing other bad side effects.

Debugging Memory Issues

X maintains a pool of memory for GUI applications. As a consequence, it will often appear that X is using a lot of memory, when in fact it is one or more applications that are consuming the memory. top may not indicate which application is to blame, but use of xrestop or other X memory display tools may help indicate it.

As a workaround, you can limit the amount of RAM X is allowed to allocate via ulimit -m in your X startup script.

Troubleshooting Common Error Messages

Common Intel Driver Error Messages

   (II) intel(0): [drm] removed 1 reserved context for kernel
   (II) intel(0): [drm] unmapping 8192 bytes of SAREA 0xf89c1000 at 0xb7b65000

These appear only on system shutdown, and generally don't indicate an issue.

   (EE) intel(0): I830 Vblank Pipe Setup Failed 0

This is because the X driver calls the DRM_I915_SET_VBLANK_PIPE ioctl after de-initializing the DRM. It should be harmless.

   (II) AIGLX: Suspending AIGLX clients for VT switch
   Error in I830WaitLpRing(), timeout for 2 seconds

This is a generic error indicating that the GPU locked up. It could be caused by a variety of issues.