This page is about the next generation test tracker, based on the hardware certification website.

The test tracker and certification website development now follows the Ubuntu release cycle with weekly roll-outs and freezes around major milestones. Work has started merging functionality from the ISO tracker into the certification site and opening the resulting site up for community use. The new site will accepts submissions of automated report submissions starting in the Jaunty cycle.

Development

Features/changes should be associated with bugs or blueprints. Where a feature/change involves database changes, those changes should be captured as a migration script (bi-directional where possible) stored in the branch.

Staging

[Proposed staging procedure:]

• changes can be pushed from trunk to staging by any developer except for Tuesdays before the release meeting provided:
  • it's been smoke tested on a local machine and
  • database migration scripts accompany code changes where needed
  • frozen on Monday before the 16h30 UTC meeting
  • updated on Tuesday after the 16h30 UTC meeting

Testing

[Describe our use of unit tests, page tests and manual smoke testing]

Production Roll-out

On a weekly (Tuesday) basis the a release meeting will be held to review the undeployed changes on staging and a decision is taken as to the revision we are going to push to production. HenrikOmma serves as release manager at these meeting, making the final call on which updates get released. The basis for making that decision should include:

• code review - have all the changes been reviewed by at least one other person

  • [can the reviews be captured in revision control?]

• completeness - commits/merges to trunk should be complete, working code including database changes

Once the decision has been made, the chosen revision is merged from staging into the production branch with an appropriate commit message.

The production branch is deployed on the production server at the end of the meeting and some basic smoke testing is done.

Production Roll-out procedure

Roll-out consists of the following steps:

• A pre-roll-out backup is taken of the code and databases
• The server is updated with the latest revision of the production branch
• Migration scripts are executed, in order, to update the database
• QA - standard (to be defined) checks are carried out as well as testing of deployed changes/features

Roll-back

If an error is encountered during the roll-out or one of the QA checks fails then the application and database are rolled back to pre-roll-out backup.

Freezes

During times of high volume of testing (milestones, particularly Beta and Release Candidates) the decision may be taken to skip deployment for a week. The primary cause for this decision would be lack of or poor quality of release reports.