KernelMaverickBIOSTestAutomation

Differences between revisions 1 and 5 (spanning 4 versions)
Revision 1 as of 2010-04-26 10:49:39
Size: 2584
Editor: eth0
Comment:
Revision 5 as of 2010-04-29 09:12:08
Size: 4209
Editor: eth0
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
 * '''Launchpad Entry''': UbuntuSpec:kernel-maverick-bios
 * '''Created''':
 * '''Launchpad Entry''': UbuntuSpec:kernel-maverick-bios-test-automation
 * '''Created''': colin-king
Line 10: Line 10:
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. See also CategorySpec for examples. It is desirable to be able automatically test specific BIOS functionality, such as suspend/resume, hibernate, wakeup, fan control, battery, C states etc.. to locate and hence be able to fix or workaround BIOS/ACPI errors. Also, it is desirable to add more kernel debug into hibernate/suspend code paths to help automatically to pin point BIOS errors. We propose a tool + kernel debug to do this automatic testing + diagnosis.
Line 20: Line 20:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. It is useful to be able to automatically detect and diagnose ACPI/BIOS issues that cause problems with a range of problems suspend/hibernate/resume, wakealarm wakeups, fan control, battery sensing, lid open/close sensing, broken ACPI hotkey events, processor C states, etc.
Line 24: Line 24:
Bill's laptop hangs when coming out of hibernate. He is unsure if the problem is a kernel driver hang or a fundamentally broken BIOS. He runs the BIOS test tool and gets a diagnosis that the BIOS does not return control back to the kernel on resume from hibernate and is directed to check for a BIOS upgrade.

Doug's laptop battery discharges without getting any feedback on the desktop. Running the BIOS test checks battery charging/un-charging and sanity checks the ACPI DSDT and informs him that there is a bug in the BAT ACPI methods.

Eve's laptop hotkeys and lid open/close events don't seem to work. The BIOS test tool detects that ACPI is not handling these events are not being correctly generated by ACPI and hence the bug is not in the kernel or userspace.

Fred's suspend/resume fails and the BIOS test tool automatically runs through a set of test scenarios and is able to detect that the BIOS works OK and the failure in in a binary blob driver when coming out of resume.

A new machine comes onto the market - running the full set of tests validates the machine and picks up some subtle ACPI semantic bugs (such as not detecting DSDT mutex wait timeouts) that may need fixing.
Line 25: Line 35:
Line 28: Line 39:
You can have subsections that better describe specific parts of the issue. Several points of attack for testing BIOS:

1. Parsing the BIOS ACPI tables, e.g. the DSDT can allow top level syntax and semantic checking using the iasl assembler/disassembler.

2. Automated test suite to test individual BIOS related issues.

3. Add extra debug hooks into the kernel suspend/resume and hibernate paths to aid debugging.
Line 32: Line 50:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: Tests in test suite will be written if possible in a high level scripting language such as Python since most of the tests can be driven from the /proc or /sys interfaces.
Line 34: Line 52:
=== UI Changes ===

Should cover changes required to the UI, or specific UI that is required to implement this

=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.

=== Migration ===

Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
Addition suspend/resume/hibernate debug code needs to be added to the kernel to aid debugging. This should be enabled via kernel boot options.

Summary

It is desirable to be able automatically test specific BIOS functionality, such as suspend/resume, hibernate, wakeup, fan control, battery, C states etc.. to locate and hence be able to fix or workaround BIOS/ACPI errors. Also, it is desirable to add more kernel debug into hibernate/suspend code paths to help automatically to pin point BIOS errors. We propose a tool + kernel debug to do this automatic testing + diagnosis.

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.

Rationale

It is useful to be able to automatically detect and diagnose ACPI/BIOS issues that cause problems with a range of problems suspend/hibernate/resume, wakealarm wakeups, fan control, battery sensing, lid open/close sensing, broken ACPI hotkey events, processor C states, etc.

User stories

Bill's laptop hangs when coming out of hibernate. He is unsure if the problem is a kernel driver hang or a fundamentally broken BIOS. He runs the BIOS test tool and gets a diagnosis that the BIOS does not return control back to the kernel on resume from hibernate and is directed to check for a BIOS upgrade.

Doug's laptop battery discharges without getting any feedback on the desktop. Running the BIOS test checks battery charging/un-charging and sanity checks the ACPI DSDT and informs him that there is a bug in the BAT ACPI methods.

Eve's laptop hotkeys and lid open/close events don't seem to work. The BIOS test tool detects that ACPI is not handling these events are not being correctly generated by ACPI and hence the bug is not in the kernel or userspace.

Fred's suspend/resume fails and the BIOS test tool automatically runs through a set of test scenarios and is able to detect that the BIOS works OK and the failure in in a binary blob driver when coming out of resume.

A new machine comes onto the market - running the full set of tests validates the machine and picks up some subtle ACPI semantic bugs (such as not detecting DSDT mutex wait timeouts) that may need fixing.

Assumptions

Design

Several points of attack for testing BIOS:

1. Parsing the BIOS ACPI tables, e.g. the DSDT can allow top level syntax and semantic checking using the iasl assembler/disassembler.

2. Automated test suite to test individual BIOS related issues.

3. Add extra debug hooks into the kernel suspend/resume and hibernate paths to aid debugging.

Implementation

Tests in test suite will be written if possible in a high level scripting language such as Python since most of the tests can be driven from the /proc or /sys interfaces.

Addition suspend/resume/hibernate debug code needs to be added to the kernel to aid debugging. This should be enabled via kernel boot options.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.


CategorySpec

KernelTeam/Specs/KernelMaverickBIOSTestAutomation (last edited 2010-05-27 09:06:01 by port-92-198-19-50)