FAQ

This page is part of the daily release process documentation

The FAQ is divided in multiple sequences depending on your role in the development of ubuntu, with the hope that you will be able to find what you are looking for quicker this way. If you have any question that isn't addressed below, please ping didrocks on IRC freenode (#ubuntu-unity) and I'll complete this page.

Upstream developers

What's needed to be done when proposing a branch or reviewing?

As discussed in DailyRelease/UpstreamMerge#The_merge_procedure, when proposing a branch or reviewing a peer's work, multiple rules have been established. The developers needs to:

• Design needs to acknowledge the change if there is any visual change involved

• Both the developer, reviewer and the integration team ensure that ubuntu processes are followed (UI Freeze/Feature Freeze for instance). If exceptions are required, they check before approving for merging that they are acknowledged by different parts. The integration team can help smooth to have this happened, but the requests should emerge from developers.

• Relevant bugs are attached to the merge proposal. This is useful for tracking what changed and for generating the changelog as we'll see in the next part
• Review of new/modified tests (for existence)
• They ensure that it builds and unit tests are passing [automated]
• Another important one, especially when you refactor, is to ensure that integration tests are still passing
• Review of the change itself by another peer contributor
• If the change seems important enough to have a bug report linked, ensure that the merge request is linked to a bug report (and you will get all the praise in debian/changelog as well!)
• If packaging changes are needed, ping the integration team so that they acknowledge them and ensure the packaging changes are part of the merge proposal.

When you approve a change, do not forget to check that there is a commit message in launchpad provided and to set the global status to "approved".

When do changes need to land in a coherent piece?

Some changes can happen and touch various components (even across different stacks). To avoid loosing a daily release because only half of transition landed in some upstream projects (and so, tests will fail because they catch this issues, isn't it? ;)), it's asked to ensure that all transitions are merged by 00 UTC. Then, you can start approving again transition starting from 06 UTC. We can try to make this window shorter in the future, but let's see first how this work in practice. For any big transition, please coordinate with the ubuntu-unity team team so that they can give a hand.

I have a change involving packaging modifications

As told in a previous section, just ask for the ubuntu-unity team to do some review or assisting in doing those changes.

I want to build the package locally myself

If you have done some packaging changes, or added a C symbol, you maybe want to ensure that it's building fine. Quite simple, just go into your branch directory and run $ bzr bd (from the bzr-builddeb package). This will build your package on your machine using the same flags and checks than on the builders. No need to autoreconf if you are using autotools, everything is handled for you. You will get warned as well if some installed files are missing.

I need to break an API/ABI

The first question is: "are you sure?". Remember that API changes are a PITA for all third parties, in particular if your API is public. Retro-compatibility is key. And yes, your dbus interface should be considered as part of your API!

If you still want to do that, ensure that you bump your soname if the ABI is broken, then, multiple packaging changes are needed. Consequently, ensure to ping your lovely ubuntu-unity team to request assistance. The transition should happen (if no retrocompatibility is provided) within the previously mentionned timeframe to have everything landing in coherence.

Also, please do think to bump the version in debian/changelog as well (this will be for next release). If there is already an UNRELEASED content, change it from

0.42.1daily83.13.09-0ubuntu1 (or 0.42.1-0ubuntu1)

to:

0.42.2-0ubuntu1 for instance (you don't need to strip the daily part, if you do, the daily release will readd it at next successful release)

Or run $ dch -i and change the version to match this (ensure you have UNRELEASED content and not "raring" or any version in the first line).

Finally, follow for all reverse dependencies the following sections about packaging B needing to be rebuilt against an ABI breakage in A.

I'm exposing a new C symbols in my library, it seems that some packaging changes are needed…

Debian packages have a debian/<packagename>.symbols file which lists all exposed symbols from your library (we only do that for C libraries as C++ does mangle the name per architecture). When you try to build the package containing a new symbol, you will see something like:

--- debian/libdee-1.0-4.symbols (libdee-1.0-4_1.0.14-0ubuntu2_amd64)
+++ dpkg-gensymbolsvPEBau       2013-02-05 09:43:28.529715528 +0100
@@ -4,6 +4,7 @@
  dee_analyzer_collate_cmp@Base 0.5.22-1ubuntu1
  dee_analyzer_collate_cmp_func@Base 0.5.22-1ubuntu1
  dee_analyzer_collate_key@Base 0.5.22-1ubuntu1
+ dee_analyzer_get_type@Base 1.0.14-0ubuntu2
  dee_analyzer_new@Base 0.5.22-1ubuntu1
  dee_analyzer_tokenize@Base 0.5.22-1ubuntu1
  dee_client_get_type@Base 1.0.0

The diff shows that debian/libdee-1.0-4.symbols doesn't list the exposed dee_analyzer_get_type new symbol. You see a version number next to it, corresponding to the current package version. The issue is that you don't know what will be the package version in the future when the next daily release will happen (even if you can infer), what to put then?

The answer is easier than you might think. As explained in the corresponding section, the daily release bot will know what version is needed, so you just need to give a hint that the new symbol is there for the upstream merger to pass.

For than, just include the symbol, with 0replaceme (0replacemepleasepleaseplease or anything else appended to 0replaceme is accepted as well if you feel adventurous ;)) as the version for the branch you will propose as a merge request. You will thus set in your symbols file:

dee_analyzer_collate_cmp@Base 0.5.22-1ubuntu1
dee_analyzer_collate_cmp_func@Base 0.5.22-1ubuntu1
dee_analyzer_collate_key@Base 0.5.22-1ubuntu1
dee_analyzer_get_type@Base 0replaceme
dee_analyzer_new@Base 0.5.22-1ubuntu1
dee_analyzer_tokenize@Base 0.5.22-1ubuntu1
dee_client_get_type@Base 1.0.0

The dee_analyzer_get_type@Base 0replaceme line will be replace with the exact version we release in the next daily release.

Also, please do think to bump the version in debian/changelog as well (this will be for next release). If there is already an UNRELEASED content, change it from

0.42.1daily83.13.09-0ubuntu1 (or 0.42.1-0ubuntu1)

to:

0.42.2-0ubuntu1 for instance (you don't need to strip the daily part, if you do, the daily release will readd it at next successful release)

Or run $ dch -i and change the version to match this (ensure you have UNRELEASED content and not "raring" or any version in the first line).

Finally, follow for all reverse dependencies the following sections about packaging B depending on a new symbol you just added on A.

My package B depends on a new symbol I just added on A or needs to be rebuilt against an ABI breakage in A

As you bumped the version in debian/changelog for the previous package, this is going to be easy.

Normally, you should have, in package B (debian/control)

Build-depends: <…>
               A (>= 0.42)

Just bump the version to the one you just added for A (normally without the -0ubuntu1). So, let's see you added a new symbol in 0.42.2-0ubuntu1:

Build-depends: <…>
               A (>= 0.42.2)

And propose that along with your change to depends on the new feature.

Think as well that for interpreted languages, you have runtime dependencies on the binary package. Imagine a new feature added on autopilot and the changelog has been bumped to 0.43:

So that unity depens at *runtime* (only for interpreted languages) on the right autopilot package:

Package: unity-autopilot
Depends: autopilot (>= 0.43)

Always think about dependencies when changing your code!

My name deserves to be in the changelog!

I'm sure you want the world to know what great modifications you have introduced. To get this praise (and blame ;)), we strongly advise you to link your change to a bug. This can be done in multiple ways, like link bugs to a merge proposal before it's approved (you link a branch to the bug report in launchpad), or use bzr commit --fixes lp:XXXX so that automatically links it for you when proposing the merge for reviewing, or put in a commit message something like this fixes bug…. This will avoid the changelog to have only empty content.

I invite you to read Why not including all commits in debian/changelog? below to understand why we don't include everything, but only bug reports title in the changelog.

I have a big change that needs to be mention in debian/changelog, but not bug for it (or I didn't link to a bug before merging

First, know that if you are wanting to link your branch to a bug after it's merged, it's too late! The daily release won't pick it up as it has know way to know it's linked to it after the fact.

You need then mention it manually in debian/changelog: If there is already an UNRELEASED content (top line in debian/changelog), add your content with:

$ dch -a

-> edit the change and save.

If it's something like "raring" or something else:

$ dch -i

-> edit the change and save.

This is creating an UNRELEASED entry for you. Check that you don't have "raring" but UNRELEASED in the first line (make the change yourself before saving if it's not the case).

I'm bumping the upstream version, what should I do?

Depends if your version is released or not.

If you bump from 1.7 to 1.8 and it's released, then to edit debian/changelog:

dch -v 1.8-0ubuntu1

If you bump from 1.7 to 1.8, but it's not released (just a pre-release bump)

dch -v 1.8~-0ubuntu1

Ubuntu Maintainers

I want to upload a change to a package under daily release process

Multiple possibilities there.

The change itself can wait next daily release

If it's an upstream change, we strongly advise you to follow the same procedure and rules than the upstream developers, meaning, proposing a merge request and so on… bzr lp-propose and bzr branch lp:<source_package_name> are regularly and automatically adjusted as part of the daily release process to point to the right branches so that you can find them easily. Vcs-Bzr in debian/control should point to the right location as well.

Then, just refer to the upstream merging guidelines above. Once approved, this change will be in the next daily release (if tests pass).

An urgent change is needed

If the change needs to be done right away, but can wait for a full testing round (~2 hours), you can:

• Propose your merge request

• Ping someone from the ubuntu-unity team who will ensure the branch is reviewing in priority and rerun a daily release manually including your change. That way, your change will get the same quality reviews and tests checking than anything else entering the distro.

It's really urgent

You can upload right away your change then, the next daily will be blocked for that component though (and only for that component) until your change reaches upstream. So please, be a good citizen and avoid more churn in proposing your change back to the upstream repo (including changelog), pinging the ubuntu-unity team preferably.

I'm making a change, should I provide anything in debian/changelog?

Not really, as mentioned earlier, if you link to a bug or mention in a commit message a bug number in your branch before proposing for review, this will be taken into account and debian/changelog will be populated automatically with your name as part of the next daily release. You can still provide it manually and the daily release will ensure there is no duplication if it was already mentionned.

If you provide it manually, just: dch -i and ensure you have UNRELEASED for an unreleased version to ubuntu (without direct upload).

ubuntu-unity team

Note that in all following commands, if -r <release> is not provided, "head" is assumed. Also, ensure you have your credentials working and be connected to the VPN (the QA team is responsible for providing those jenkins credentials).

Where is the jenkins start page?

There it is.

Forcing a stack publication

After reviewing and ensuring that we can release a stack (see section about manual publication causes, like packaging changes to review or upstream stack failing/in manual mode). Then run:

$ cu2d-run -P <stack_name> -r <release>

example for indicators, head release:

$ cu2d-run -P indicators

Remember that the master "head" job is not rerun, so it will stay in its current state (unstable/yellow). The publish job should though goes from unstable/yellow to green if everything went well.

Rerun a full stack daily release, rebuilding everything

$ cu2d-run -R <stack_name> -r <release>

Rerun a partial stack daily release, rebuilding only some components

$ cu2d-run -R <stack_name> -r <release> <components>

this will keep the state and take into account (failure to build state for instance) and the binaries packages for the other components which were part of this daily release, but not rebuilt.

for instance, keeping dee and libunity, but rebuilding compiz and unity from the unity, head release stack: $ cu2d-run -R unity compiz unity

The integration tests will still take everything we have (so including dee and libunity) at the latest stack with the new trunk content for compiz and unity when we relaunched this command.

Rerun integration tests, but taking all ubuntu-unity ppa's content (like transition needing publication of the indicator and unity stacks)

$ cu2d-run -R <stack_name> -r <release> --check-with-whole-ppa

This command doesn't rebuild anything, just run the "check" step, but with the whole ppa content (basically doing a dist-upgrading instead of selecting only desired components from this stack). This is handy when for instance a transition involved the indicator and unity stack. So the indicator integration tests failed (we don't have the latest unity stack built yet). Unity built and tests pass. To be able to release the indicator stack as well as the unity stack, we need the indicator stack to publish, for that, we relaunch only the indicator tests, but with the whole ppa content, meaning last Unity.

In that case, the publish for the indicators will be manual (you will need to force the publication) and as well, you should ensure that you will publish the unity stack at the same time as well to have everything in coherence copied to the -proposed pocket. More information on the stack dependency [DailyRelease/StackDependencies|in this section].

I know that I want for one time skipping one arch where my package won't built for

If one arch is build-dep wait and we know that it won't get unblock for this daily, it's possible to skip that over for this run using: $ cu2d-skip -r <release> stack project [archs_to_skip]

archs_to_skip is a space separated value mentionning every archs we wait the wait-for-ppa to skip. Remember this is a one time thing. For longer-term, the packaging needs to be fixed to tell on what arch it's building its binaries packages (a detection is done when initializing the watcher).

Adding/removing components to a stack

The stack files are living in the stack/ folder of lp:cupstream2distro-config. This yaml file structure should be straightforward. Note that the tuple component: branch_location is optional (it will be set to lp:component if not provided). So add/change what you need in those file.

Be aware that all running jobs are stopped for this stack.

$ cu2d-update-stack -U <path_to_stack_file>

This will reset/reconfigure as well all branches, add -S if you don't have access to them (but ensure someone will configure them still at least once)

Then, ping an archive admin so that he pulls the modification on lillypilly for the second-safety check process.

The archive admin needs to, after checking that lp:cupstream2distro only adds component for filtering we want:

• $ ssh lillypilly.canonical.com
  $ sudo -u ubuntu-archive -i
  $ cd cu2d/cupstream2distro/
  $ bzr pull

The server side filtering will be refreshed then.

Different level of failures acceptance

We do accept some integration tests failing (no unit tests should fail though as ran as part of package building). The different levels of those triggers are defined in a file like daily-release/config/autopilot.rc. Those are different level of failures, regressions, skipped and removed tests per stack. The real file depends on stackname and are found in ~jenkins/cu2d/<stack_name>.autopilotrc as described in the -check template.

Monitoring upstream uploads and merges

Remember to regularly look at the -changes mailing list to pick if a manual upload has been done and port the changes back (if not proposed by the ubuntu developer making the upload) to the upstream branch. The component which had a manual upload is ignored for daily release until this backport is done, meaning that no new commits will be published until then (you can see the prepare job for that component will turn unstable).

Also, ensure as you are watching all upstream merges to follow our acceptance criterias that the "latestsnapshot" branches generated automatically by the daily release process has been merged successfully by the upstream merger.

Boostrapping a new component for daily release

A lot of information is available on this wiki page.

General infos

Why not including all commits in debian/changelog?

We made the choice to only mention changes associated bugs. This also mean that if upstream never link bugs to merge proposal, or use "bzr commit --fixes lp:XXXX" or put in a commit message something like "this fixes bug…", the upload will have an empty changelog (just having "latest snapshot from rev …") and won't detail the upload content.

This can be seen as suboptimal, but I think that we should really engage our upstream to link fixes to bugs when they are important to be broadcasted. The noise of every commit message to trunk isn't suitable for people reading debian/changelog, which is visible in update-manager. If people wants to track closely upstream, they can look at the bzr branch for that. You can see debian/changelog as a NEWS file, containing only important information from that upload. We need of course to have more and more people upstream realizing that and being educated that linking to a bug report is important.

If an information is important to them but they don't want to open a bug for it if there isn't one, there is still the possibility, as part of the upstream merge, to directly feed debian/changelog with a sensible description of the change.

Versionning scheme

We needed to come with a sane versionning schema. The current one is: # based on current upstream version, like 6.5 # appending "daily" to it # adding the current date in yy.mm.dd format So we end up with, for instance: 6.5daily13.01.24. Then, we add -0ubuntu1 as the packaging is separated in the diff.gz by using split mode.

If we need to rerun the same day a release for the same component, as we detect that a version was already released in Ubuntu or eventually in the ppa, this will become: 6.5daily13.01.24.1, and then 6.5daily13.01.24.2… I think you got it

We can use this general pattern: <upstream_version>daily<yy.mm.dd(.minor)>-0ubuntu1 for the package. The next goal is to have upstream_version as small (one, two digits?) as possible.