ArmSuspendResumeTesting

Launchpad Entry: mobile-lucid-arm-suspend-resume-testplan
Created: 2009-11-02
Contributors: AlexanderSack, PaulLarson
Packages affected:

Summary

Suspend/Resume is an essential use-case for netbook/smartbook users and as such requires extensive testing during development cycle and before release. Hence, this spec is supposed to adds extensive suspend/resume testing to the arm image and hardware testing efforts. This spec identifies required testcases, prioritizes them and defines abstract actions as applicable.

Rationale

By extending and improving the checkbox tests for suspend/resume we can ease the ability to do suspend/resume testing.

Design

The implementation section defines testcases, assigns priorities to them and gives background where available. We will allocate one action for each testcase that will deal with doing the research and filling in the missing details of this specification.

The goal is to get the testcases implemented and merged into checkbox - preferably fully automated and in a way that non-arm architectures might also benefit from this work in the future.

Implementation

Testcase	Summary	Priority	Status	Comments
memory/ECC	compare /proc/meminfo before/after resume	Base	DONE	automated
network	ping after resume still working	Base	DONE	automated
graphics	no artifacts or glitches in graphics after resume	Base	DONE	manual
HID devices	verify input devices working after resume, also remove or attach during sleep	Base	DONE	manual: mice, accelerometers, touchscreen, electrostatic button bars, fingerprint and other auth devices
cpufreq/voltage	run something before/after resume, make sure cputime for execution is reasonably close	Base	DONE
usb/usb otg	add and remove device during sleep	STD	DONE	block devs, not char devs which are covered under HID
media playback	suspend during audio/video playback, should resume from where it left off	STD	DONE
SD card/SDIO	remove during sleep	STD	DONE
wireless soft switch	compare rfkill list before/after, ask if nm reset correctly	STD	DONE
keyboard soft switches	capslock, numlock, etc should be in same state as before suspend	STD	DONE
v4l	video capture devices should still function after resume, suspend during capture may be an interesting test as well	Extra
multi-display	connect second display, xrandr, second display should still be active after resume, also disconnecting while sleeping should not cause problems	Extra
bluetooth adapters/devices	NEEDINFO	Extra
pcie	NEEDINFO	Extra
PCMCIA	NEEDINFO	Extra
eSATA	remove device during suspended state	Extra
network file systems	suspend/resume while mounted	Extra
network block devices (nbd, iSCSI, etc)	suspend/resume while mounted	Extra
multicore tests	NEEDINFO all cores should still be in the same state, processes pinned to cpus should be on the same core, etc	Extra
Mobile radio/telephony devices	NEEDINFO	Extra
fan/cooling	functioning correctly on resume	Extra
Encryption Engines	NEEDINFO	Extra
NAND/MTD	NEEDINFO	Extra		should only be relevant during boot, not on resume. persia may have some ideas here
backlight	restore to previous state	Extra

Notes/Gobby Transscript

https://blueprints.edge.launchpad.net/ubuntu/+spec/mobile-lucid-arm-suspend-resume-testplan

https://wiki.ubuntu.com/KernelTeam/SuspendResumeTesting 300 consecutive suspend/resume uses wake alarm scripts RTC wakeup

http://kernel.ubuntu.com/git?p=cking/scripts/.git;a=summary

Works for freescale, works for marvell

How can you make sure that every driver has reinitialized properly on resume? * something has to be done on each resume to test them all * could add a lot of manual tests to do this * existing test scripts do not currently cover this * drivers have to be checked after suspend/resume and hibernate

- memory / ECC - suspend resume during media playback (audio/video) - v4L - network could be checked with ping - graphics will need manual look
- - xrandr - second display - disconnect display
- usb / USB OTG
- remove card during sleep (manual test)
- Bluetooth adapters + devices - connectable buses - PCI-express, PCMCIA, eSATA - audio tests? state can be saved and compared - manjo will cover these cases - network file systems - network block (iSCSI and ata over ethernet) - ext2/3/4 fs - raw MLC NAND and UBIFS is likely to be used considering its low price - sd card test / SDIO
- remove card during sleep
- HID devices
- - mouse - accelerometers - touchscreen - electrostatic button bars (persia) - fingerprint readers / authentication devices
- cpufreq / voltage resume to previous state / full function
- - backlight - fan / cooling - sensors? (may not be right place)
- include multicore (for future) - OEM GPIO/etc?
- - soft switches for wireless control - soft switches / keys for keyboard media functions
- Encryption Engines - Mobile / Telephony devices - Liquid on Lucid

Discussion about the implications of suspend/resume on local media playback

- expected to resume where it left off after resume
- it is possible using known power management hooks to prevent sleep or ConsoleKit to pause media before suspend and play media after resume

ACTIONS:

Identify what of the ones above can be tested automatically
Gather input from QA/OEM team about what is already done
Identify what parts can be added to checkbox
Identify what parts are really special to mobile
Ensure that things that are not mobile specific are getting done too (not necessarily by mobile team).
Check what is different for hibernation (compared to suspend)

Two QA leads, people will have to volunteer to test some of these subsystems Write up test cases so that volunteers can run them Write up automated test cases - QA