CloudinitUpdates

Differences between revisions 1 and 10 (spanning 9 versions)
Revision 1 as of 2017-02-01 17:49:46
Size: 1682
Editor: powersj
Comment:
Revision 10 as of 2018-09-17 09:17:58
Size: 6251
Editor: chad.smith
Comment: [Removed dependecy on MAAS QA feedback for cloud-init SRUs because Solutions QA team exercises the MAAS datasource during testing: reviewed rbasak]
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
'''DRAFT DRAFT DRAFT''' This document describes the policy for updating cloud-init in a stable, supported release.
Line 3: Line 3:
This document describes the policy for updating Cloud-init in a stable supported distro, including LTS. cloud-init is initialization software used for standing up cloud instances. It can help setup, configure, and customize an instance during boot.
Line 5: Line 5:
Cloud-init is... In order to closely align with the MAAS product and the needs of cloud providers (e.g. AWS, Azure, GWS) cloud-init needs to be periodically updated in order to enable new features. Therefore, the following types of changes are allowed as long as the processes outlined below are followed:

   * Bug fixes
   * New features

In the event of a change breaking backwards compatibility, then SRU team approval will need to be obtained by emailing the ubuntu-release team mailing list.

== Requesting the SRU ==
The SRU should be done with a single process bug, instead of individual bug reports for individual bug fixes. The one bug should have the following:

   * The SRU should be requested per the StableReleaseUpdates documented process
   * The template at the end of this document should be used and all ‘TODO’ filled out
   * The change log will contain a reference to the single SRU process bug, not all bugs fixed by the SRU. However, if there are very important bugs that are deemed worthy of reference they too should be included in the change log.
   * Major changes should be called out in the SRU template, especially where changed behavior is not backward compatible.
   * For each release (e.g. Ubuntu 14.04, Ubuntu 16.04, etc.) that is proposed to be updated by the SRU a link to the results of integration testing for at least the following datasources must be provided:
     * nocloud (e.g. kvm)
     * lxd
     * ec2 (e.g. aws)
     * azure
     * gce
   * Any architecture specific fixes need to be noted and architecture specific test results included
   * Any packaging changes (e.g. a dependency change) need to be stated
   * If any manual testing occurs it should also be documented. See [[ http://launchpad.net/bugs/1588052 | LP: #1588052 ]] as an example.
Line 8: Line 30:
=== Merge QA ===
This is the mandatory QA process that the proposed packages have to pass. The following requirements must be met:
=== Merges ===
Updates to cloud-init trunk go through the following process:
Line 11: Line 33:
   * Every change needs to be covered by a Launchpad bug to track all changes
   * Every bug must have:
      * A bzr branch attached with the fix + changes covered by unit tests
      * Link to successful run of integration tests based on the bzr branch
      * Reviewed and approved by a member of the development team
      * Branch set to the committed state
   * All unit tests and style tests ran and passed successfully
   * Code coverage of changes or code coverage % either no change or increased(?)
   * Reviewed and approved by a member of the development team
   * Daily integration tests on trunk
   * Successful run of unit tests and style tests based on the branch
   * Branch set to the committed state
Line 20: Line 38:
=== Packaging QA ===
TBD
=== Packaging ===
The following describes the requirements for each package generated for the SRU.
Line 23: Line 41:
== Requesting the SRU ==
The SRU should be requested per the StableReleaseUpdates documented process.
For each package generated a successful completion of cloud-init integration tests, as described below, using the proposed package with no unexplained errors or failures
Line 26: Line 43:
The SRU should be done with a single process bug for this stable release exception, instead of individual bug reports for individual bug fixes. Individual bugs may be referenced in the change log, but in that case each of those bugs will need to be independently verified and commented on for the SRU to be considered complete. === Integration Tests ===
Integration testing involves two seperate sections: automated and manual.
Line 28: Line 46:
The description of the bug should contain links to automatic testing results (e.g. Jenkins test runs) so that anyone can verify the testing that occurred and the results. Additionally, the SRU bug should be verbose in documenting any manual testing that occurs. See [[https://bugs.launchpad.net/ubuntu/+source/snapd/+bug/1588052 | LP# 1588052]] as an example ==== Automated Tests ====
Results from the automated test cases using the version from proposed, against all releases need to be attached. The automated test cases cover a variety of cloud-config based scenarios to ensure changes to cloud-init do not introduce regressions or unnecessary changes in behavior.
Line 30: Line 49:
=== SRU Template ===
TBD
These tests are run against the LXD and KVM backend today. Because of this lack of coverage against other datasources the following manual test are also required.

==== Manual Tests ====
Integration testing involves taking the proposed version of cloud-init and running it against a specific test case. Integration testing needs to take place across all updated releases and a variety of supported platforms. Releases tested should involve all releases expected to be updated. Supported platforms must contain at least each of the following:

   * nocloud (e.g. kvm)
   * lxd
   * ec2 (e.g. aws)
   * azure
   * gce

The test case should be developed as a part of each resolved bug or new feature. This way testing is straightforward and clear as to what is expected to work.

=== Curtin Testing ===
The curtin vmtest should also be sucessfully ran using cloud-init from proposed and results attached.

=== Solutions Testing ===
Due to the dependency on cloud-init with various other products, the solutions testing team will run their continuous integration test against the cloud-init that is in -proposed. A successful run for each specific release will be required before the proposed cloud-init can be let into -updates.

The cloud-init team will be in charge of attaching the artifacts and console output of the appropriate run to the bug. cloud-init team members will not mark ‘verification-done’ until this has happened.

== SRU Template ==
{{{
== Begin SRU Template ==
[Impact]
This release sports both bug-fixes and new features and we would like to
make sure all of our supported customers have access to these
improvements. The notable ones are:

   * <TODO: Create list with LP: # included>

See the changelog entry below for a full list of changes and bugs.

[Test Case]
The following development and SRU process was followed:
https://wiki.ubuntu.com/CloudinitUpdates

The cloud-init team will be in charge of attaching the artifacts and
console output of the appropriate run to the bug. cloud-init team
members will not mark ‘verification-done’ until this has happened.

* Automated Test Results
<TODO: attach automated cloud-init-proposed test artifacts from tests for each release with lxd artifacts>
<TODO: attach automated cloud-init-proposed test artifacts from tests for each release with kvm artifacts>
<TODO: attach automated curtin vmtest with cloud-init proposed>
<TODO: attach Solutions Testing team test results for each LTS>

* Manual Test Results
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on ec2 datasource>
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on gce datasource>
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on azure datasource>

[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.

[Discussion]
<TODO: other background>

== End SRU Template ==

<TODO: Paste in change log entry>
}}}

== Past SRUs ==
Links to past SRUs using this process are below:

* TBD

This document describes the policy for updating cloud-init in a stable, supported release.

cloud-init is initialization software used for standing up cloud instances. It can help setup, configure, and customize an instance during boot.

In order to closely align with the MAAS product and the needs of cloud providers (e.g. AWS, Azure, GWS) cloud-init needs to be periodically updated in order to enable new features. Therefore, the following types of changes are allowed as long as the processes outlined below are followed:

  • Bug fixes
  • New features

In the event of a change breaking backwards compatibility, then SRU team approval will need to be obtained by emailing the ubuntu-release team mailing list.

Requesting the SRU

The SRU should be done with a single process bug, instead of individual bug reports for individual bug fixes. The one bug should have the following:

  • The SRU should be requested per the StableReleaseUpdates documented process

  • The template at the end of this document should be used and all ‘TODO’ filled out
  • The change log will contain a reference to the single SRU process bug, not all bugs fixed by the SRU. However, if there are very important bugs that are deemed worthy of reference they too should be included in the change log.
  • Major changes should be called out in the SRU template, especially where changed behavior is not backward compatible.
  • For each release (e.g. Ubuntu 14.04, Ubuntu 16.04, etc.) that is proposed to be updated by the SRU a link to the results of integration testing for at least the following datasources must be provided:
    • nocloud (e.g. kvm)
    • lxd
    • ec2 (e.g. aws)
    • azure
    • gce
  • Any architecture specific fixes need to be noted and architecture specific test results included
  • Any packaging changes (e.g. a dependency change) need to be stated
  • If any manual testing occurs it should also be documented. See LP: #1588052 as an example.

QA Process

Merges

Updates to cloud-init trunk go through the following process:

  • Reviewed and approved by a member of the development team
  • Daily integration tests on trunk
  • Successful run of unit tests and style tests based on the branch
  • Branch set to the committed state

Packaging

The following describes the requirements for each package generated for the SRU.

For each package generated a successful completion of cloud-init integration tests, as described below, using the proposed package with no unexplained errors or failures

Integration Tests

Integration testing involves two seperate sections: automated and manual.

Automated Tests

Results from the automated test cases using the version from proposed, against all releases need to be attached. The automated test cases cover a variety of cloud-config based scenarios to ensure changes to cloud-init do not introduce regressions or unnecessary changes in behavior.

These tests are run against the LXD and KVM backend today. Because of this lack of coverage against other datasources the following manual test are also required.

Manual Tests

Integration testing involves taking the proposed version of cloud-init and running it against a specific test case. Integration testing needs to take place across all updated releases and a variety of supported platforms. Releases tested should involve all releases expected to be updated. Supported platforms must contain at least each of the following:

  • nocloud (e.g. kvm)
  • lxd
  • ec2 (e.g. aws)
  • azure
  • gce

The test case should be developed as a part of each resolved bug or new feature. This way testing is straightforward and clear as to what is expected to work.

Curtin Testing

The curtin vmtest should also be sucessfully ran using cloud-init from proposed and results attached.

Solutions Testing

Due to the dependency on cloud-init with various other products, the solutions testing team will run their continuous integration test against the cloud-init that is in -proposed. A successful run for each specific release will be required before the proposed cloud-init can be let into -updates.

The cloud-init team will be in charge of attaching the artifacts and console output of the appropriate run to the bug. cloud-init team members will not mark ‘verification-done’ until this has happened.

SRU Template

== Begin SRU Template ==
[Impact]
This release sports both bug-fixes and new features and we would like to
make sure all of our supported customers have access to these
improvements. The notable ones are:

   * <TODO: Create list with LP: # included>

See the changelog entry below for a full list of changes and bugs.

[Test Case]
The following development and SRU process was followed:
https://wiki.ubuntu.com/CloudinitUpdates

The cloud-init team will be in charge of attaching the artifacts and
console output of the appropriate run to the bug.  cloud-init team
members will not mark ‘verification-done’ until this has happened.

* Automated Test Results
<TODO: attach automated cloud-init-proposed test artifacts from tests for each release with lxd artifacts>
<TODO: attach automated cloud-init-proposed test artifacts from tests for each release with kvm artifacts>
<TODO: attach automated curtin vmtest with cloud-init proposed>
<TODO: attach Solutions Testing team test results for each LTS>

* Manual Test Results
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on ec2 datasource>
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on gce datasource>
<TODO: attach manual cloud-init-proposed test artifacts from tests for each release on azure datasource>

[Regression Potential]
In order to mitigate the regression potential, the results of the
aforementioned integration tests are attached to this bug.

[Discussion]
<TODO: other background>

== End SRU Template ==

<TODO: Paste in change log entry>

Past SRUs

Links to past SRUs using this process are below:

* TBD

CloudinitUpdates (last edited 2019-01-25 18:26:38 by chad.smith)