DebuggingKernelSuspend

Debugging Central

This page is part of the debugging series — pages with debugging details for a variety of Ubuntu packages.

Introduction

This page describes how to debug suspend to RAM, and resume from suspend to RAM problems with your computer.

For issues regarding suspend to disk (i.e. hibernate), please see here.

Suspend and resume use facilities within your BIOS called ACPI, or Advanced Configuration and Power Interface. Linux provides an ACPI subsystem that manages the suspend and resume process. Usually problems occur when resuming, and normally the culprit is a device driver that does not recover from a powered down state. If your computer successfully performs a suspend, then it is quite likely any resume problems are due to another device driver and not the ACPI subsystem.

The debugging procedure described below assumes you have already installed the latest BIOS from your vendor, and requires a linux kernel with the capability of "/sys/power/pm_trace". You can ensure that pm_trace'ing is possible by looking in the directory /sys/power/ (from a terminal type ls /sys/power/) to see if a file called pm_trace exists.

Debugging information to provide in your bug report

In order to identify the root cause of the issue, all of the following information needs to be provided, uncompressed and untarred, in order for a developer to work on it.

  1. Which part of the process does the issue occur with, the suspend to ram, or resuming from?
  2. Please advise how you suspended specifically. For example:
    • Executing at a terminal pm-suspend
    • Shutting the lid of your laptop, which is set to suspend on close.
    • Clicking the word Suspend in the GUI.
    • The computer suspends automatically on inactivity.
  3. Please advise how you resumed specifically. For example:
    • Pressed the power button
    • Lifted the lid
    • Clicked a key on the keyboard
  4. While booted into the latest non-daily mainline kernel, execute at a terminal:

    cat /proc/acpi/wakeup > wakeup

    Please attach to your report the resume trace mentioned below.

  5. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo freezer > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  6. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo devices > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  7. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo platform > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  8. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo processors > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  9. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo core > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  10. While booted into the latest non-daily mainline kernel, execute at a terminal:

    sudo su
    echo none > /sys/power/pm_test
    exit

    Please attach to your report the resume trace mentioned below.

  11. Please provide the output of the following terminal command:

    cat /sys/kernel/debug/suspend_stats 
  12. If you have a graphics related issue after resume (corruption, display blank, etc.) please SSH into your machine and capture both /var/log/Xorg.0.log and /var/log/Xorg.0.log.old. Please attach each separately.

"resume-trace" debugging procedure for finding buggy drivers

Resume problems are difficult to debug. The approach used here needs to make notes on progress during resume and be able to recover them after a manual reboot. But there is no non-volatile storage available at the time resume is bringing up your computer. The only hardware on a PC motherboard that retains information across power cycles is the real time clock (RTC), so that is what is used. For those that want to know the details, please read Documentation/power/s2ram.txt in your kernel sources, or see here. The implementation of suspend/resume debug trace is in drivers/base/power/trace.c.

Caveat Emptor: Using the following debug suggestions will radically change the values in your RTC chip, so much so that your file system will think it has been eons since the last fsck. You can avoid a long fsck delay by using 'tune2fs'. For example, 'tune2fs -i 0 /dev/sda1' disables fsck on boot. But first you'll want to use 'tune2fs -l <partition>' to find your current settings - look at the "Check interval" setting.

From a fresh install, the default time interval is 15552000, which is 6 months.

Remember that when you are all done, you'll want to set your RTC clock back. Note that NTP by itself is not sufficient to recover the correct time of day, since by default NTP is configured to do nothing if the time is off by more than 1000 seconds. You must first use the 'ntpdate' or 'date' command to get the clock close.

In order to simulate your suspend/resume process, enter the following commands:

sudo sh -c "sync && echo 1 > /sys/power/pm_trace && pm-suspend"

At this point your computer should enter the suspend state within a few seconds. Usually the power LED will slowly flash when in the suspended state. When that has happened, initiate the resume process by pressing the power button. Observe closely if the disk light comes on briefly. This indicates that resume has begun. If resume fails to complete, then press the power button until the computer turns off. Power on your computer making sure that it loads the same kernel that exhibited the resume problem. You have about 3 minutes to start this boot process before the information saved in the RTC gets corrupted.

Start a console and enter:

dmesg > dmesg.txt

You can edit this file and find lines similar to these:

[   11.323206]   Magic number: 0:798:264
[   11.323257]   hash matches drivers/base/power/resume.c:46

There may well be another 'hash matches' line beyond that. If so, then you are in luck because the last one is the likely culprit. For example:

hash matches device i2c-9191

The only way to prove this is to remove the module prior to initiating suspend. Repeat as needed.

If you get a device number rather than a name, lspci and /sys/devices/pci* are your friends.

As well, please attach your dmesg.txt to your bug report.

Further reading

Detailed analysis of ACPI kernel code for debugging a suspend problem


Also see:


CategoryBugSquad CategoryDebugging

DebuggingKernelSuspend (last edited 2016-08-02 10:01:23 by penalvch)