= Process to promote an image with failing tests =
The purpose of this process is to be able to promote an image with failing tests, when the test has been identified as a non-regression, and fix these unstable tests to make them reliable.

 1. Get a list of failures from the CI dashboard
 1. For each failure on the dashboard
    1. CI file a bug against the project with the failure
    1. CI either run the test manually or ask dev (test must be run by hand) <<FootNote(manually: In this context manually means running the test step by step on a device without autopilot or any form of automation.)>>
    1. QA documents manual steps if not already documented. <<FootNote(QA will document the tests for the failures that occurred during the last 7 days as one off to establish the base line.)>>
 1. If --
    1. test passes in manual execution:
      1. CI retargets the bug against the infrastructure <<FootNote(infrastructure: CI and Autopilot)>> [QA or CI owns / investigates]
      1. Promote the build if there is no other regression
    1. test fails in manual execution:
      1. CI retargets the bug against the production code/application [Dev owns / looks]
      1. Reject the build


== Questions ==

 * Q: What is the incentive for people to fix broken tests? 
 * A: Give people 2 weeks to fix the test, after this time, block the build

== Landing email changes ==

 * Include bug number for every test with problems
 * date of each bug 
 * time it has been open
 * before an issue is removed from the landing email, QA verifies how this has been fixed.