CorePackages

Differences between revisions 1 and 20 (spanning 19 versions)
Revision 1 as of 2012-08-15 19:58:18
Size: 4027
Editor: c-71-236-173-142
Comment: OpenStack Packaging Workflow in Ubuntu
Revision 20 as of 2016-08-29 16:43:57
Size: 5391
Editor: corey.bryant
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
<<TableOfContents()>> ## page was renamed from OpenStack/Packaging
## page was renamed from OpenStack/PackagingWorkflow
## page was renamed from OpenStack/Packaging
Line 3: Line 5:
= Packaging Branches = == OpenStack Core Packages - How to update, build, and contribute ==
Line 5: Line 7:
== Ubuntu Server Dev == === Overview ===
Line 7: Line 9:
Packaging for the main Ubuntu packages can be tracked in the branches of [[http://launchpad.net/~ubuntu-server-dev|the Ubuntu Server Dev Team]]. Branches names are formatted as lp:~ubuntu-server-dev/$openstack-component/$openstack-release, ie: lp:~ubuntu-server-dev/nova/folsom. Note that branches for the current development OpenStack release will map to the packaging for the current Ubuntu development release (Folsom -> Quantal). Ubuntu maintains it own set of Bazaar branches and Git repositories for !OpenStack core packages on Launchpad.
Line 9: Line 11:
== Proposed Packaging Branches == !OpenStack releases prior to Liberty are [[https://code.launchpad.net/~ubuntu-server-dev|maintained in Bazaar]], Liberty onwards are [[https://code.launchpad.net/~ubuntu-server-dev/+git|maintained in Git]].
Line 11: Line 13:
The majority of the packaging work we do takes place in the -proposed branches of [[http://launchdpad.net/~openstack-ubuntu-testing|OpenStack Ubuntu Testing Team]] and are formatted as lp:~ubuntu-server-dev/$openstack_component/$ubuntu_release-$openstack_release-$proposed, ie: lp:~openstack-ubuntu-testing/nova/precise-folsom-proposed. Changes made to these packaging branches get immediately incorporated into our automated package builds and integration testing. This page discusses the Git workflow. The Bazaar work flow is [[https://wiki.ubuntu.com/ServerTeam/OpenStack|discussed here]].
Line 13: Line 15:
== Contributing == === Git Configuration ===
Line 15: Line 17:
Packaging bugs should be filed in Launchpad and marked as affecting the corresponding Ubuntu package. First, you'll want to setup git to simplify your work-flow. See 'Configuring Git' in the [[https://help.launchpad.net/Code/Git|Launchpad git documentation]].
 
=== Tools ===
Line 17: Line 21:
To contribute packaging fixes, base your changes on corresponding branch in the Proposed Packaging Branches--not the Ubuntu Server Dev or main Ubuntu archive branches. The branch may contain a changelog entry with an 'UNRELEASED' series name. Please document your changes within that block and provide any relevant bug numbers. If approved, a member of the [[http://launchdpad.net/~openstack-ubuntu-testing|OpenStack Ubuntu Testing Team]] will merge the change into the -proposed branch. After merging, the change will get coverage in CI (automated builds, integration testing, PPA uploads) and subsequent upload into the Ubuntu Archive and/or Cloud Archive. The !OpenStack packaging git repositories all make use of git-buildpackage and pristine-tar for managing upstream release tarballs.
Line 19: Line 23:
For security reasons, membership to the [[http://launchdpad.net/~openstack-ubuntu-testing|OpenStack Ubuntu Testing Team]] is restricted, however, individuals who show sustained, quality contribution may be considered. For a general overview of git-buildpackage and pristine-tar, see the Debian [[http://wiki.debian.org/PackagingWithGit|documentation]]. This wiki page details the specifics of how we use these tools in Ubuntu.
Line 21: Line 25:
= Continious Integration = === Branches ===
Line 23: Line 27:
For updated status of current package builds and deployment tests, see [[https://jenkins.qa.ubuntu.com/view/Openstack%20Testing/view/Overview/|Ubuntu QA Jenkins Dashboard]] The current development release packaging is maintained under the 'master' branch; released versions of !OpenStack will mirror upstream branch management - i.e. stable/liberty.
Line 25: Line 29:
= PPAs = === Cloning a git repository ===
Line 27: Line 31:
The resulting binary packages from the automated CI builds get uploaded to various PPAs depending on the upstream release and branch. If you are running on the latests Ubuntu release, then the following should dtrt:
Line 29: Line 33:
'''Testing packages are provided for testing and evaluation use only. They come without support or guarantees.''' {{{
debcheckout --git-track='*' <projectname>
}}}
Line 31: Line 37:
== Folsom Trunk Testing PPA == for example
Line 33: Line 39:
We are currently building Folsom packages for both Ubuntu 12.04 and 12.10. Packages are uploaded to different release pockets in single PPA. This PPA should serve as a installation source on both Ubuntu releases for Folsom testing packages. These PPAs may lag behind upstream branches depending on the state of PPA build queues or any pending build faiulres.

 * [[https://launchpad.net/~openstack-ubuntu-testing/+archive/folsom-trunk-testing|ppa:openstack-ubuntu-testing/folsom-trunk-testing]]

In addition to the core Openstack packages, this PPA is also kept up-to-date with new dependencies that are either making their way into the main Ubuntu archive (for 12.10 and beyond) or will be satisfied by the Cloud Archive (in the case of 12.04 LTS).

== Essex/Stable Testing PPA ==

Packages are built for new commits to the stable/essex branches of each core component in Essex. These are uploaded to a PPA to be used for smoke testing the stable/essex branch before pushing updates into the [[https://wiki.ubuntu.com/StableReleaseUpdates|Ubuntu SRU Queue]]

 * [[https://launchpad.net/~openstack-ubuntu-testing/+archive/essex-stable-testing|ppa:openstack-ubuntu-testing/essex-stable-testing]]

= Ubuntu Cloud Archive =

~+ Please see [[https://wiki.ubuntu.com/ServerTeam/CloudArchive|The Ubuntu Cloud Archive]] +~

= More Info =
{{{
debcheckout --git-track='*' nova
}}}
Line 52: Line 44:
[[http://www.openstack.org/blog/2012/02/automating-openstack-testing-on-ubuntu/|Automating OpenStack Testing on Ubuntu]] Using the git-track option ensures that all required branches are automatically setup - for example:

{{{
$ git branch
* master
  pristine-tar
  upstream
}}}

The 'upstream' and 'pristine-tar' branches are used by git-buildpackage to manage the upstream release tarballs.

If you're not on the latest Ubuntu release then directly clone the repository and checkout the branches:

{{{
git clone lp:~ubuntu-server-dev/ubuntu/+source/nova
git checkout pristine-tar
git checkout upstream
git checkout master
}}}

=== Building a source package ===

Use the buildpackage subcommand of gbp:

{{{
gbp buildpackage -S
}}}

To build without signing the .changes/.dsc files:

{{{
gbp buildpackage -S -us -uc
}}}

The source package will be places in ../build-area.

=== Building Debian packages ===

We generally use sbuild to build debian packages.

Steps for building an !OpenStack Newton package for yakkety:

{{{
mk-sbuild yakkety
sbuild-update -udc yakkety-amd64
sbuild -A -d yakkety-amd64 <path-to-dsc-file>
}}}

For initial setup steps see: https://wiki.ubuntu.com/SimpleSbuild

Steps for building an !OpenStack Newton package for xenial:

{{{
sudo add-apt-repository ppa:ubuntu-cloud-archive/tools
sudo apt update
sudo apt install cloud-archive-utils
mk-sbuild xenial
sbuild-update -udc xenial-amd64
sbuild-newton -A -d xenial-amd64 <path-to-dsc-file>
}}}

=== Making a packaging change ===

Make your packaging changes as you normally would under the debian folder; then use 'git add' to add the changed files to the staging area; if you have added an appropriate changelog comment then:

{{{
debcommit
}}}

will DTRT and annotate the git commit with the changelog entry details - you get a chance to edit this if you have provided anything other than a single line entry to the changelog.

=== Updating to a new release ===

First, grab the new upstream release:

{{{
uscan --verbose --rename (and optionally: --download-version <version>)
}}}

this will drop the orig.tar.gz into the parent folder of the git repository. Then use git-buildpackage to import the archive:

{{{
gbp import-orig ../nova_12.0.0~b1.orig.tar.gz
}}}

This action will import the content into the upstream branch, create the required pristine-tar data and place it in the pristine-tar branch, and then merge the upstream branch into the master branch (or stable branch).

=== Maintainers - pushing updates to Launchpad ===

To push any local commits and tags to Launchpad:

{{{
git push --all
git push --tags
}}}

'''NOTE''': you need to be a member of the ubuntu-server-dev team to be able todo this - this includes all Ubuntu Core Developers.

=== Contributors - pushing a personal repository to Launchpad ===

To push any local commits and tags to Launchpad:

{{{
git push --all lp:~<launchpad-id>/ubuntu/+source/nova
git push --tags lp:~<launchpad-id>/ubuntu/+source/nova
}}}

=== Contributors - proposing a change ===

There are a few ways to propose a package change:

1. Reference your git repo in the bug report:

{{{
lp:~<launchpad-id>/ubuntu/+source/nova
}}}

2. Attach a debdiff to the bug report:

{{{
debdiff nova_current_version.dsc nova_new_version.dsc > nova_new_version.debdiff
}}}

3. Create a launchpad merge proposal:

Navigate to the URL of your pushed repository:
https://code.launchpad.net/~<launchpad-id>/ubuntu/+source/nova/+git/nova

Propose each branch separately:
 * Click on the branch name and 'Propose for merging'
 * Select the target repository: lp:~ubuntu-server-dev/ubuntu/+source/nova
 * Select the target reference path: master, upstream, or pristine-tar

OpenStack Core Packages - How to update, build, and contribute

Overview

Ubuntu maintains it own set of Bazaar branches and Git repositories for OpenStack core packages on Launchpad.

OpenStack releases prior to Liberty are maintained in Bazaar, Liberty onwards are maintained in Git.

This page discusses the Git workflow. The Bazaar work flow is discussed here.

Git Configuration

First, you'll want to setup git to simplify your work-flow. See 'Configuring Git' in the Launchpad git documentation.

Tools

The OpenStack packaging git repositories all make use of git-buildpackage and pristine-tar for managing upstream release tarballs.

For a general overview of git-buildpackage and pristine-tar, see the Debian documentation. This wiki page details the specifics of how we use these tools in Ubuntu.

Branches

The current development release packaging is maintained under the 'master' branch; released versions of OpenStack will mirror upstream branch management - i.e. stable/liberty.

Cloning a git repository

If you are running on the latests Ubuntu release, then the following should dtrt:

debcheckout --git-track='*' <projectname>

for example

debcheckout --git-track='*' nova

Using the git-track option ensures that all required branches are automatically setup - for example:

$ git branch
* master
  pristine-tar
  upstream

The 'upstream' and 'pristine-tar' branches are used by git-buildpackage to manage the upstream release tarballs.

If you're not on the latest Ubuntu release then directly clone the repository and checkout the branches:

git clone lp:~ubuntu-server-dev/ubuntu/+source/nova
git checkout pristine-tar
git checkout upstream
git checkout master

Building a source package

Use the buildpackage subcommand of gbp:

gbp buildpackage -S

To build without signing the .changes/.dsc files:

gbp buildpackage -S -us -uc

The source package will be places in ../build-area.

Building Debian packages

We generally use sbuild to build debian packages.

Steps for building an OpenStack Newton package for yakkety:

mk-sbuild yakkety
sbuild-update -udc yakkety-amd64
sbuild -A -d yakkety-amd64 <path-to-dsc-file>

For initial setup steps see: https://wiki.ubuntu.com/SimpleSbuild

Steps for building an OpenStack Newton package for xenial:

sudo add-apt-repository ppa:ubuntu-cloud-archive/tools
sudo apt update
sudo apt install cloud-archive-utils
mk-sbuild xenial
sbuild-update -udc xenial-amd64
sbuild-newton -A -d xenial-amd64 <path-to-dsc-file>

Making a packaging change

Make your packaging changes as you normally would under the debian folder; then use 'git add' to add the changed files to the staging area; if you have added an appropriate changelog comment then:

debcommit

will DTRT and annotate the git commit with the changelog entry details - you get a chance to edit this if you have provided anything other than a single line entry to the changelog.

Updating to a new release

First, grab the new upstream release:

uscan --verbose --rename  (and optionally: --download-version <version>)

this will drop the orig.tar.gz into the parent folder of the git repository. Then use git-buildpackage to import the archive:

gbp import-orig ../nova_12.0.0~b1.orig.tar.gz

This action will import the content into the upstream branch, create the required pristine-tar data and place it in the pristine-tar branch, and then merge the upstream branch into the master branch (or stable branch).

Maintainers - pushing updates to Launchpad

To push any local commits and tags to Launchpad:

git push --all
git push --tags

NOTE: you need to be a member of the ubuntu-server-dev team to be able todo this - this includes all Ubuntu Core Developers.

Contributors - pushing a personal repository to Launchpad

To push any local commits and tags to Launchpad:

git push --all lp:~<launchpad-id>/ubuntu/+source/nova
git push --tags lp:~<launchpad-id>/ubuntu/+source/nova

Contributors - proposing a change

There are a few ways to propose a package change:

1. Reference your git repo in the bug report:

lp:~<launchpad-id>/ubuntu/+source/nova

2. Attach a debdiff to the bug report:

debdiff nova_current_version.dsc nova_new_version.dsc > nova_new_version.debdiff

3. Create a launchpad merge proposal:

Navigate to the URL of your pushed repository: https://code.launchpad.net/~<launchpad-id>/ubuntu/+source/nova/+git/nova

Propose each branch separately:

  • Click on the branch name and 'Propose for merging'
  • Select the target repository: lp:~ubuntu-server-dev/ubuntu/+source/nova
  • Select the target reference path: master, upstream, or pristine-tar

OpenStack/CorePackages (last edited 2022-02-28 15:39:15 by corey.bryant)