Coverity

Differences between revisions 5 and 7 (spanning 2 versions)
Revision 5 as of 2012-02-08 22:00:16
Size: 7037
Editor: c-98-212-2-38
Comment:
Revision 7 as of 2012-02-08 22:01:28
Size: 7028
Editor: c-98-212-2-38
Comment:
Deletions are marked like this. Additions are marked like this.
Line 27: Line 27:
 * '''Code Snippet''': the source-file interleaved with Coverity's defect-report.  * '''Code Snippet''': an excerpt from the source-file interleaved with Coverity's defect-report.
Line 47: Line 47:
Developers and managers will use [[ #BugsInLaunchpad | Launchpad bugs ]] to track Coverity defects. Coverity scans will run as part of a Continuous Integration process, and defects will be submitted to Launchpad from Coverity by a '''sync script''' (described below). We envision a few different paths for a Coverity defect through the Ubuntu developer process--in detail: Developers and managers will use Launchpad bugs to track Coverity defects. Coverity scans will run as part of a Continuous Integration process, and defects will be submitted to Launchpad from Coverity by a '''sync script''' (described below). We envision a few different paths for a Coverity defect through the Ubuntu developer process--in detail:

Introduction

The product strategy team in side of Canonical has licensed Coverity to help increase the quality of various projects that we are working on. Each commit to the trunk of the projects that fall under the Unity umbrella will be scanned by the Coverity Static Analyzer to determine defects in the codebase. These defects can range from NULL pointer dereferencing to more complex memory issues.

Coverity is a proprietary tool that we unfortunately can't distribute to the community, but we have built tools to publish the information that Coverity provides in Launchpad so that everyone in the greater Unity community can see the results of the runs. We expect both Canonical developers and community developers' primary interaction with Coverity to be through the Launchpad bugs, but Canonical developers do have access to the server itself. If someone in the community does need more information that isn't exported on a bug, please contact a Canonical developer and we'd be happy to get that for you. If you believe that a piece of information should consistently be exported and you know Python, patches to the sync script are welcome as well.

Bugs in Launchpad

Coverity tracks a specific issue through a variety of transformations that it could go through. This includes through difference series of the code base, or as the code moves throughout the project. It also merges defects from discrete series into a single bug.

Tags

We're using a variety of bug tags to give extra information about the bug and how it relates to Coverity.

  • coverity -- this tag marks the bug as being found by the Coverity static analyzer

  • coverity-checker_name -- there are a variety of different checks that the analyzer does, and each of them is named by the type of error that it finds. These tags mark the type of error that Coverity has found and can be used for filtering bugs. For more information on checkers see the dictionary of checkers.

Description

We use the description field to give quick information about the issue that Coverity found and some structured data to be able to track back to the Coverity tracker.

  • CID: the Coverity Unique ID in the Coverity tracker. This is irrelevant except for tracking back to the Coverity database, which could sometimes be required.

  • Checker: the 'checker' Coverity has used to find this defect.

  • CWE Definition: a link to the defect's CWE/MITRE definition.

  • File: the source file in which the defect was found.

  • Function: the function in which the defect was found. Note that not all defects are found in a proper function.

  • Code Snippet: an excerpt from the source-file interleaved with Coverity's defect-report.

Status

The Coverity sync tool will create new bugs with status of "New". It is expected that the developers on the project will adjust the status to track the bug all the way through to the "In Progress" state. When there is a commit to the trunk of the project and Coverity is able to verify the fix it will set the state to "Fix Committed". It is expected that the release manager for the project would then move the status of the bug to "Fix Released" when the bug is released by the project. If Coverity detects that a bug has reemerged in the codebase it will change it from "Fixed *" back to "New."

Severity

The priority is the bug is set initially by the script using a set of metrics based on the checker as provided by Coverity. Mostly this relates to the severity of what happens when the bug is hit. The Coverity sync script does not change the severity if it is modified by developers, it is expected they may reprioritize based on area of code or whether this bug is important to the development team. More information can be found in the dictionary of checkers and their severities.

Attachments

On each bug there are also several attachments which provide more information about the error that Coverity has found. This can include things like extra information on which conditionals are being used and where there are perhaps other errors that may contribute. These are largely free form in their description of the error and designed to provide more information to the developer. Individual developers may prefer to use this information differently.

Tasks and Series

Coverity will look to understand which series of the project that it is parsing and can link the various defects across those series. It will maintain the status on the series target for the bug.

Workflow

Developers and managers will use Launchpad bugs to track Coverity defects. Coverity scans will run as part of a Continuous Integration process, and defects will be submitted to Launchpad from Coverity by a sync script (described below). We envision a few different paths for a Coverity defect through the Ubuntu developer process--in detail:

  • New -> Confirmed -> In Progress -> Fix Committed -> Fix Released

    • The vanilla workflow: Coverity discovers a defect and reports it as a bug in Launchpad. A developer fixes the bug as part of his/her normal workflow, ultimately merging his/her branch into trunk. When Coverity fails to find this defect in a new scan, the bug is marked "Fix Committed". When the fixed code is released, a release manager marks the bug as "Fix Released".
  • New -> Confirmed -> In Progress -> New

    • Coverity discovers a defect and reports it as a bug in Launchpad. A developer fixes the bug as part of his/her normal workflow, ultimately merging his/her branch into trunk. Coverity finds that the defect still exists, and re-opens the Launchpad bug by re-setting its status to "New", and the triage/fixing process begins again.
  • New -> Invalid/Wontfix

    • Coverity discovers a defect and reports it as a bug in Launchpad. A developer believes the bug to be "Invalid" or marks it "Wontfix". Coverity's successive discovery of the same defect won't result in a new bug report.

Sync Script

There is launchpad project for the Launchpad Coverity Sync Script where the source can be viewed, branched and reviewed by anyone. Unfortunately, we can't provide a test setup or a way for you to test builds without buying a Coverity license, but the code is available and we're happy to help test changes.

How it works

The script goes through a few different conditions as explained in this graphic:

bug work flow

CanonicalProductStrategy/Coverity (last edited 2015-07-08 16:02:36 by 1)