Ubuntu Server Team
#ubuntu-server on Freenode IRC
The Ubuntu Server team works to enable and promote the use of Ubuntu Server, the number one cloud OS.
We specifically focus on three key areas:
- Providing a robust and stable infrastructure for scale-out computing deployments.
- Supporting the latest scale-out computing workloads and architectures.
- Providing the right tools for orchestrating services within a scale-out computing environment.
If you want to contact the ServerTeam use the following resources:
Join our mailing list at https://lists.ubuntu.com/mailman/listinfo/ubuntu-server.
You can read an archive of messages at https://lists.ubuntu.com/archives/ubuntu-server/.
The server team utilizes IRC to offer support for server-related questions. The team sits on freenode in the #ubuntu-server channel.
The ubottu IRC bot makes it easy to share an extensive set of factoids to others in an IRC channel. E.g. typing !ask | noobie will cause ubottu to tell noobie that folks should just go ahead and ask their questions. Ubottu can also conveniently show the channel information on bugs and packages. See ubottu for more details.
There are different areas where you can help the Ubuntu Server Team. Here are a few ideas:
Help on Mailing List & IRC
You can lend a hand with people's questions and problems on the above mailing list and the IRC channel. Ask and answer questions, provide suggestions, and provide input in our periodic calls for input.
Test, Test, Test
If you have server-type hardware or can spin up VMs and containers, you can make sure that Ubuntu is supported and works well on it. You can also test software and features worked on by the Ubuntu Server Team.
You can head to the community documentation to check that server related pages are up to date or help to get the Ubuntu Server Team wiki pages into shape. You can also help with the Ubuntu Server Guide, the official Server documentation.
Whenever we fix a bug in past stable releases, we need somebody who is not the developer making the fix to verify that the package fixes and doesn't cause any obvious regressions. It is vital that we have people test these updates before they are sent to all users.
The full process for doing so is here:
Here is the list of server bugs needing verification.
The goal of triage is to move bugs that are in a NEW status to a CONFIRMED or INVALID status.
To get started:
Choose a bug from the New,Unconfirmed bug list and subscribe to it.
- Work with the user to identify the issue using the INCOMPLETE status.
- Once the bug is reproducible and the process to reproduce it is clearly documented in the bug thread set the status to CONFIRMED. If it turns out that there isn't any bug set the status to INVALID.
- You can unsubscribe from the bug as your role as a bug triager is finished.
- Read more bug triaging resources:
You can have a look at the list of packages looked after by the Ubuntu Server Team to see if some needs packaging work.
This is a excellent way to gain experience to become a MOTU.
Ubuntu Server Bug Triage
Goal: To successfully review every bug filed against Ubuntu Server related packages
A review involves analyzing a bug to determine if the bug is valid and if sufficient information was provided. If the bug is both valid and provided with sufficient information, the bug is marked as triaged and will be worked to closure by a member of the server team. Otherwise, the bug will be responded to and marked as 'Incomplete' for more details, 'Invalid' for not a real bug, or 'Won't Fix'. In certain (rare) cases bugs might stay in new/confirmed which reflects we need to look into it again more deeply to triage in a better way.
This is a list of the various queues of interest to the server team. They are a good place to go if you are looking for a good "what to do next" bug.
Bug triage is completed for all bugs last updated on a particular day. An assigned member of the server team will look at all bugs that were updated on the previous day. For example, the member with responsibility on Friday will review all the bugs updated on Thursday. For the weekend, the member with responsibility on Monday will review all the bugs updated on Friday, Saturday, and Sunday.
This process is expected to take less than 90 minutes per day. This is not meant to be a full root cause analysis (RCA) investigative time, instead only determining if further attention is warranted and sufficient information has been provided.
If a bug is considered to be a benefit to the majority of users and needing further attention by the Ubuntu Server Team the triager subscribes ~ubuntu-server to it. If possible a task is scheduled e.g. on https://trello.com/b/U9HhWyT0/daily-ubuntu-server for the team or by the triager subscribing/assigning himself to look into it later if possible. Please do note that the classic consideration of a bug being "actionable" is flagged place in the launchpad bug status. We might in some cases e.g. only want to track the bug and therefore subscribe ~ubuntu-server. But On the other hand a triager has to be clear that a totally non-actionable bug is likely to appear as expiring 180 days later, so if possible drive the bugs further via clarifications, questions, more bug tasks, subscribing subject matter experts and so on.
Current members of Ubuntu Server bug triage: https://launchpad.net/~ubuntu-server-active-triagers/+members
Direct team subscriptions
We subscribe ~ubuntu-server directly to a bug to track our community bug backlog when the bug meets the following criteria. When the bug no longer meets these criteria, we unsubscribe it:
- Anything that, if the bug turns out to valid, is something that would be under the ~ubuntu-server remit to fix (common use cases but not obscure ones - but nothing stops an individual volunteering to work on an obscure use case, of course).
- By definition, if it's something that we wouldn't fix and request volunteers even if we had time, then it doesn't warrant a subscription.
- This subscription is for the Ubuntu Server triage community and is not for tracking of internal Canonical customer requests. Whether a Canonical customer has made a request in relation to a particular bug makes no difference and provides no additional priority under this process. A Canonical customer bug may still be subscribed if it qualifies under these criteria.
- If the bug is assigned to someone on our team, leave it subscribed. No need to subscribe, and feel free to unsubscribe.
Since the backlog is bigger than what can be achieved in a short time, there is the extra classification via the tag server-next. That tag is set by the Triager (or anyone else working on doing the Root-Cause-Analysis or a Fix) to reflect that this is an issue that shall be tackled by the Teams resources "next".
Another reason to add server-next in some cases is to preserve high quality contributions of the community. An example might be a report that the user already bisected and created a patch for - in those cases the benefit diminishes by bit rot way too fast, so handling that next helps to retain the work the reporters did. And vice versa it might encourage one or the other to provide more high quality bugs.
The goal is to have this list around ~20 bugs most of the time, if dropping below we can refill with candidates from the ~ubuntu-server subscribed bugs. But if it grows significantly out of this range it is non-realistic to expect those issues to be handled in time, we should communicate so to the reporters.
The rules of the server-next tag are as follows:
- Must not tag unless bug is actionable. Doesn't mean it must have a patch, only that a developer has enough information to work on the bug, even if it means more debugging.
- Tag only if one of these two things are true:
- Delays will discourage this excellent community contribution.
- If you believe it affects a major use case for Ubuntu server users. In this case you should also set the bug Importance.
- The set of all bugs tagged server-next must be kept small. If it grows, the lowest priority bugs tagged server-next must be removed until the list isn’t too big.
- This tag is for the Ubuntu Server triage community and is not for tracking of internal Canonical customer requests. Whether a Canonical customer has made a request in relation to a particular bug makes no difference and provides no additional priority under this process. A Canonical customer bug may still be tagged if it qualifies under these criteria.
- If the bug is assigned to or otherwise owned by someone on our team, there is no need to tag it.
- Remove the tag when the bug is assigned to or otherwise owned by someone on our team.
Daily Bug Expiration
There are two levels of expiration. The tooling will help to report these to the Triager. Eventually we want those lists of expiring bugs to be zero, and if they are not at least clear them on the day they expire.
Note: especially initially these lists are rather huge and working through all of them takes various subject matter expertise, any help getting those re-triaged is highly appreciated.
Server-next expiration - default after 60 days
- If we considered a bug actionable and added it to server-next, but then no update happened in 60 days that usually means something went wrong. Often bugs are blocked on external constraints. This needs to be evaluated as a case-by-case decision. Most common cases are, that it turns out:
that the bug is not solvable/reasonable the way it was planned -> re-triage, maybe drop server-next
that it is actually fixed or otherwise progressed without update -> update bug
that we failed to give it the required focus -> add the server-triage-discuss tag to the bug and bring it in the next standup
Server subscription expiration - default after 180 days
- If nobody touched a bug for 180 days (~= 1 release cycle) it is reasonable to check for changed conditions. Quite often e.g. a patch one was waiting on is available now. In other cases a newer release fixed it already. Essentially anything that is listed here needs to be fully re-triaged to ensure the list is reflecting the current status. It also can after this time be used as a metric how many more people chimed in got dupped on the bug (importance/#affected). Most common cases are, that it turns out:
that recent releases upstream or even already in Ubuntu have the fix -> re-triage, consider tagging server-next for SRU
that the bug should have been supported by the community but nothing happened -> re-triage importance, consider dropping ~ubuntu-server subscription
- that a bug that was formerly considered a real case is not qualifying anymore (e.g. alternative solutions
have taken hold as "the" way to do it) -> re-triage importance, consider dropping ~ubuntu-server subscription
- if unsure, add the server-triage-discuss tag and bring it up at the next standup
Overall for all of these we have to be honest to the bug reporter, try to understand why an issue was not worked on and explain it if possible. Also if we drop server-next or the ~ubuntu-server subscrription for any of the reasons above always add a explanatory comment. If reporters disagree with our re-triage they will report on the bug and it will show up in the daily triage duty the next day to be reconsidered with that point of view taken into consideration.
To assist with the triaging there is https://github.com/powersj/ubuntu-server-triage/. By default it will list the bugs of the current days Triager, but it can be controlled via commandline arguments. That way one can easily do tasks like "last wednesdays tirage duty" if one was away a few days.
Furthermore it will list the expiring bugs that should be re-triaged.
Helpful Guides and Definitions:
Ubuntu Server Packaging
We are transitioning to a git-based workflow for handling package changes. This also allows us to use Launchpad Merge Proposals and request Reviews before publishing a new package version.
Requesting a review
There are three review teams within canonical-server:
canonical-server-core-reviewers: members are Ubuntu Core Developers and can upload any package
canonical-server-motu-reviewers: members are Ubuntu MOTU developers and can upload to Universe
canonical-server-packageset-reviewers: members are Ubuntu Server Developers and can upload Server related packages
To find out which team should review your branch, use the ubuntu-upload-permission tool from the ubuntu-dev-tools package and select the most specific team:
$ ubuntu-upload-permission --list-uploaders <package>
A second review slot should always be requested for the canonical-server team, as we want the merge proposal to always show up in the active reviews page for the canonical-server team regardless if a review slot was taken or not. Launchpad bug #1708172 was filed about it, but for now we will use this workaround.
All this translates to the following git ubuntu command:
$ git ubuntu submit --reviewer canonical-server --reviewer <reviewer team> --target-branch <target>
In this command, <target> depends on which Ubuntu release you are targeting:
ubuntu/devel: for the current development version of Ubuntu
ubuntu/<name>-devel: for the <name> Ubuntu release. For example, if preparing an SRU for Xenial, the target reference path would be ubuntu/xenial-devel.
Alternatively, you can go to https://launchpad.net/people/+me/+git, select your repository, a branch within, and then "propose for merging". The details to be filled out are:
target repository: that will be lp:~usd-import-team/ubuntu/+source/<sourcepackage>
target reference path: the same as <target> in the git ubuntu submit command above
reviewer: canonical-server-core-reviewers, canonical-server-motu-reviewers or canonical-server-packageset-reviewers. This depends on the package, see previous discussion about using the ubuntu-upload-permission tool to find out.
After proposing the merge, request a second reviewer slot for canonical-server and you are set.
Eventually you will get review comments. These can of a few types:
- just a comment. Can also be a non-formal review from someone outside the server team.
- approval (also known as a +1)
- needs information: the reviewer didn't understand something, or wants clarification
- needs fixing: the reviewer didn't agree with something and is requesting that it be changed
- disapprove: the approach to the problem is incorrect (also known as a -1)
- abstain (also known as a +0)
- resubmit: you should resubmit the proposal, possibly because the package has changed while this review was up
If you need to work on something in your merge proposal, you should set its status back to "work in progress" to signal others that you are addressing the issue. This will have the effect of removing the proposal from the queue and then people can concentrate on reviewing other branches while you work on yours. If you have Server Trello board rights, you should also move the review card back to the "Doing" list. Once you are done, place it again in the "Review Requests" list and change the status of your MP to "needs review".
Reviewing a Merge Proposal
First of all, never claim the canonical-server review slot. That slot should remain unclaimed as its sole purpose is to facilitate viewing all merge proposals in a single page. You should only claim the review slot of one of the canonical-server reviewers teams. You are welcome to add a review comment anytime, however.
If you have Server Trello board rights, once the review is claimed, move the corresponding Trello card in the server board from the "Review Requests" list to the "Review-Doing" one.
Developer & Packaging Resources
We are focusing on server related packages in main and universe. Developers can use the Triaged Ubuntu Server bugs list to prioritize their work.
For packaging information, head to the MOTUs, the Master Of The Universe.
There is the PackagingGuide.
UbuntuDevelopers explains how to become an official packager.
ubuntu-motu mailing list and #ubuntu-motu on irc.freenode.org are good places to ask for help.
For Ubuntu release specific resources, the Ubuntu Team wiki is the central location where Ubuntu developers exchange ideas and track their progress.
UbuntuDevelopment gives an overview of the development processes.
The ubuntu-devel mailing list and #ubuntu-devel on irc.freenode.net are the places where ubuntu developers can be found.
For server packages stuck in proposed from migrating to release: