MIRTeam

Differences between revisions 1 and 53 (spanning 52 versions)
Revision 1 as of 2012-05-29 20:49:53
Size: 2174
Editor: pool-173-48-209-5
Comment:
Revision 53 as of 2019-09-18 21:19:49
Size: 13815
Editor: jdstrand
Comment: add parenthetical on where the to find the golang guidelines
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
== Weekly status meeting ==

The MIR Team holds weekly meetings on Tuesdays, at 1 PM UTC, in #ubuntu-meeting.

If you're chairing the meeting, you can use http://people.ubuntu.com/~cyphermox/meetings/ubuntu-mir.html as a simple script.

Line 9: Line 16:
The primary decision a reviewer is making is "Will this package be well maintained in main?" All the rest of these guidelines are just ways to help you answer to that question. The primary decision a reviewer is making is '''"Will this package be well maintained in main?"''' The following guidelines are just ways to help you answer to that question.

=== Duplication ===

One easy way to avoid the burden of maintaining the package is to not use it in the first place! If a package is pulling in some random jpeg parsing library that needs a MIR, maybe it makes more sense to patch the package to just use libjpeg instead. Keep an eye out for duplicated functionality in main, since that makes bug fixing and security reviewing that much harder.

=== Embedded sources and static linking ===
Embedding a library source increases the maintenance burden of a package since that source needs to be maintained separately from the source in the Ubuntu archive. If a source embeds another package, in general the embedded package should not be used and the packaging should be modified to use the Ubuntu archive version. When this is not possible, the security team must agree to using the embedded source.

Similarly, when a binary from one source package statically links to libraries from another source package from the archive, when those libraries are updated the statically linked binaries must be rebuilt with the updated libraries to receive the fix, which increases the maintenance burden. For this reason, static linking in archive builds is discouraged unless static linking is required for the package in question to function correctly (eg, an integrity scanner).

==== golang ====
golang 1.4 packages and earlier could only statically compile their binaries.
golang 1.5 in Ubuntu 16.10 introduces `-buildmode=shared` to build shared libraries and
`-linkshared` to dynamically link against shared libraries. In general, statically
compiled binaries are not suitable for the Ubuntu archive because they increase the
maintenance burden significantly. As such, from Ubuntu 16.10 through 17.04, golang packages
packages in main were expected to be built with shared libraries.

Evaluating cost/benefits while considering the ABI instability of golang libraries during
this period, the MIR team decided for 17.10 and later to allow static builds of golang
packages in main, so long as the number of these packages remains low and they follow the
guidelines below (under "Packaging red flags/If it's a statically compiled golang package").
The MIR team may reevaluate this in the future.

'''IMPORTANT''': in [[http://bazaar.launchpad.net/~ubuntu-security/ubuntu-cve-tracker/master/view/head:/README.built_using|17.10+]], the requesting team must state
their commitment to testing no-change-rebuilds triggered by a dependent
library/compiler and to fix any issues found for the lifetime of the release.
Line 13: Line 47:
Determine if the package may have security implications (has a history of CVEs, runs a daemon as root, parses data formats, opens a port). Err on the side of caution. Determine if the package may have security implications (has a history of CVEs, runs a daemon as root, uses webkit^1,2^, uses lib*v8 directly, parses data formats, opens a port, processes arbitrary web content, uses centralized online accounts, integrates arbitrary javascript into the desktop, deals with system authentication (eg, pam), etc). Err on the side of caution.
Line 15: Line 49:
If the package is security sensitive, you have two options. You can either review as much as you can and then assign to the ubuntu-security team. Or you can immediately re-assign to a member of the MIR Team that is also on the Security Team (there is usually at least one). If the package is security sensitive, you should review as much as you can and then assign to the ubuntu-security team. The bug will then be added to the [[https://trello.com/b/HvFhIQpv/security-team|security team trello board]] (private board; Reviews lane) in the prioritized list of MIR security reviews.

 0. While qt5webkit is in main, [[https://bugs.launchpad.net/ubuntu/+source/qtwebkit-opensource-src/+bug/1192567/comments/1|it is temporary]]
 0. Now that [[http://www.chriscoulson.me.uk/blog/?p=196|oxide-qt]] is in main, all software needing a webengine should be ported to work with the [[https://launchpad.net/ubuntu/+spec/client-1308-oxide|oxide bindings for the chromium content API]]. Application authors are encouraged to use !UbuntuWebView, but may also use Oxide directly.
Line 21: Line 58:
 * If it's a library, does it either have a symbols file or use an empty argument to dh_makeshlibs -V?
 * If it's a Python package, does it use dh_python3?
 * If it's a Python package going on the CD, will it pull in Python 2?
 * Does it have a team bug subscriber? (poke bdmurray once a team signs on, so that he can update the [[http://reqorts.qa.ubuntu.com/reports/m-r-package-team-mapping.html|master list]] in [[http://bazaar.launchpad.net/~ubuntu-archive/ubuntu-archive-tools/trunk/view/head:/package-subscribers#L107|ubuntu-archive-tools]])
 * Is the code translatable (if user visible)?
 * If it's a Python package, does it use dh_python?
 * If it's a Python package going on the desktop CD, will it pull in Python 2?
Line 28: Line 66:
 * Does it have a bug subscriber in Ubuntu? Ideally some team in Ubuntu is looking after the package.  * If it's a library, does it either have a symbols file or use an empty argument to dh_makeshlibs -V? (pass such a patch on to Debian, but don't block on it). Note that for C++, see [[DailyRelease/FAQ]] for a method to demangle C++ symbols files.
Line 30: Line 68:
 * Is its update history slow or sporadic?
 * Is the current release packaged?
 * Will entering main make it harder for the people currently keeping it up to date? (i.e. are they only MOTUs?)
Line 31: Line 72:
 * Is debian/rules a mess? Ideally it uses dh7 and overrides to make it as tiny as possible.
 * Does debian/control use `Built-Using`. This may may indicate static linking which should be discouraged (excepting golang, see below)
 * If it's a statically compiled golang package:
  * Does the package use dh-golang (if not, suggest dh-make-golang to create the package)?
  * Does debian/control use `Built-Using: ${misc:Built-Using}` for each non'-dev' binary package (importantly, golang-*-dev packages only ship source files so don't need Built-Using)?
  * Does the package follow [[http://pkg-go.alioth.debian.org/packaging.html|Debian Go packaging guidelines]]?
Line 36: Line 83:
 * Uses of sudo or LD_LIBRARY_PATH  * Uses of sudo, gksu, pkexec, or LD_LIBRARY_PATH
 * User nobody is strictly for NFS's use and must not be used by any running processes on the system.
 * Use of setuid needs evry careful design (prefer systemd to set those for services)
Line 38: Line 87:
 * Dependency on webkit, qtwebkit, seed or libgoa-*
 * Embedded source copies (this happens frequently with Go upstreams)
 * If this is a scope for the Unity Dash, does it [[http://mhall119.com/2012/12/adding-privacy-setting-support-to-your-unity-lens/|honor the privacy settings]]?

=== Process states ===

This is supposed to be an overview of MIR bug status (as there was enough confusion in the past).
Especially to make clear who's turn it is to act next on a MIR bug.

|| 1. New/Confirmed (unassigned)|| Bug is new for the MIR team to dispatch it to a MIR Team member||
|| 2. New/Confirmed (assigned)|| on the TODO list of the assigned MIR team member||
|| 3. New/Confirmed (assigned to Security)|| on the TODO list of the Security Team||
|| 4. In Progress (any)|| MIR team ack (and if needed Security Team ack) done, but now needs the Dependency/Seed change to happen to pull it into Main||
|| 5. Fix Committed (any)|| All of the above done, waiting for an AA to promote the packages||
|| 6. Fix Released (any)|| Case resolved by an AA||
|| 7. Incomplete (any)|| Questions/Requests were raised for the bug reporter to resolve/clarify||
|| 8. Won't Fix (any)|| Final Nack or given up by the bug reporter||
|| 9. Invalid (any)|| No Feedback for a while when in incomplete status||

{{{
          Answered +-7----------+ No feedback +-9-------+
      +------------------+ Incomplete +------------------------> Invalid |
      | old assign can | any <--------------------+ | any |
      | be kept as-is +-----^------+ | +---------+
      | | question |
      | | |
+-1---v-------+ Triage +-2---+----+ Security needed +-3-+------------------+
| New +--------->+ New +----------------->+ New |
| unassigned | | assigned | | assigned to Security |
+-------------+ +-+--+--+--+ +-+--+--+--------------+
                           | | | +-8-----------+ | | |
                           | | +----> Won't Fix <----+ | |
                           | | Nack| any | Nack | |
                           | | +-------------+ | |
                           | | | |
                           | | +-4-----------+ | |
                           | +-------> In Progress <-------+ |
                           | Ack | any | Ack |
                           | +-------------+ |
                           | |Change that pulls |
                           | |into Main |
+-6------------+ | +-5-v-----------+ |
| Fix Released | +---------> Fix Committed <---------+
| any <---------------------+ any |
+--------------+ AA promotes +---------------+
}}}

Note1: Since many people set things to confirmed once they are sure it "exists" all too often bugs get set to confirmed. Since confirmed does not have any meaning for our process we will handle new<->confirmed as if the'y be the same.

Note2: All other combinations are undefined and should be fixed up to one of the defined states
Line 43: Line 143:
 * `reverse-depends` can tell you reverse source or binary depends, per component
Line 44: Line 145:

== Making Life Easier for Archive Team Members ==

To help prevent promotion of packages that cause component mismatches, we can do two things:

 1. Run `check-mir` and make sure that all dependencies have a MIR. We don't want to be surprised by a dependency after a package is promoted.
 1. List all distinct binary packages that should be promoted. Often a source package will have binary packages that aren't actually needed in main. Things like `-doc` or `-autopilot`. These can stay in universe, and it is a kindness to list only the packages we need for the archive team member that does the promotion.

== Bug Lists ==

 * [[https://bugs.launchpad.net/~ubuntu-mir|All MIR bugs]]
 * [[https://bugs.launchpad.net/~ubuntu-mir/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE|All open MIR bugs]]
 * [[https://bugs.launchpad.net/ubuntu/?field.searchtext=&orderby=-importance&field.status%3Alist=NEW&assignee_option=none&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=ubuntu-mir&field.structural_subscriber=&field.component-empty-marker=1&field.tag=&field.tags_combinator=ANY&field.status_upstream-empty-marker=1&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_no_package.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on&search=Search|All open unclaimed MIR bugs]]
 * [[https://bugs.launchpad.net/ubuntu/?field.searchtext=&orderby=-importance&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=ubuntu-mir&field.structural_subscriber=&field.component-empty-marker=1&field.tag=&field.tags_combinator=ANY&field.status_upstream-empty-marker=1|All incomplete MIR bugs]]
 * [[https://bugs.launchpad.net/%7Eubuntu-mir/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&assignee_option=choose&field.assignee=ubuntu-security&field.bug_reporter=&field.bug_commenter=&field.subscriber=ubuntu-mir&field.structural_subscriber=&field.tag=&field.tags_combinator=ANY&field.status_upstream-empty-marker=1&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on|All MIR bugs where the security team is assigned]]

Introduction

The Ubuntu MIR Team reviews packages for promotion from universe to main. See MainInclusionProcess for information on how to request an MIR.

Weekly status meeting

The MIR Team holds weekly meetings on Tuesdays, at 1 PM UTC, in #ubuntu-meeting.

If you're chairing the meeting, you can use http://people.ubuntu.com/~cyphermox/meetings/ubuntu-mir.html as a simple script.

Reviewing

The primary decision a reviewer is making is "Will this package be well maintained in main?" The following guidelines are just ways to help you answer to that question.

Duplication

One easy way to avoid the burden of maintaining the package is to not use it in the first place! If a package is pulling in some random jpeg parsing library that needs a MIR, maybe it makes more sense to patch the package to just use libjpeg instead. Keep an eye out for duplicated functionality in main, since that makes bug fixing and security reviewing that much harder.

Embedded sources and static linking

Embedding a library source increases the maintenance burden of a package since that source needs to be maintained separately from the source in the Ubuntu archive. If a source embeds another package, in general the embedded package should not be used and the packaging should be modified to use the Ubuntu archive version. When this is not possible, the security team must agree to using the embedded source.

Similarly, when a binary from one source package statically links to libraries from another source package from the archive, when those libraries are updated the statically linked binaries must be rebuilt with the updated libraries to receive the fix, which increases the maintenance burden. For this reason, static linking in archive builds is discouraged unless static linking is required for the package in question to function correctly (eg, an integrity scanner).

golang

golang 1.4 packages and earlier could only statically compile their binaries. golang 1.5 in Ubuntu 16.10 introduces -buildmode=shared to build shared libraries and -linkshared to dynamically link against shared libraries. In general, statically compiled binaries are not suitable for the Ubuntu archive because they increase the maintenance burden significantly. As such, from Ubuntu 16.10 through 17.04, golang packages packages in main were expected to be built with shared libraries.

Evaluating cost/benefits while considering the ABI instability of golang libraries during this period, the MIR team decided for 17.10 and later to allow static builds of golang packages in main, so long as the number of these packages remains low and they follow the guidelines below (under "Packaging red flags/If it's a statically compiled golang package"). The MIR team may reevaluate this in the future.

IMPORTANT: in 17.10+, the requesting team must state their commitment to testing no-change-rebuilds triggered by a dependent library/compiler and to fix any issues found for the lifetime of the release.

Security

Determine if the package may have security implications (has a history of CVEs, runs a daemon as root, uses webkit1,2, uses lib*v8 directly, parses data formats, opens a port, processes arbitrary web content, uses centralized online accounts, integrates arbitrary javascript into the desktop, deals with system authentication (eg, pam), etc). Err on the side of caution.

If the package is security sensitive, you should review as much as you can and then assign to the ubuntu-security team. The bug will then be added to the security team trello board (private board; Reviews lane) in the prioritized list of MIR security reviews.

  1. While qt5webkit is in main, it is temporary

  2. Now that oxide-qt is in main, all software needing a webengine should be ported to work with the oxide bindings for the chromium content API. Application authors are encouraged to use UbuntuWebView, but may also use Oxide directly.

Common blockers

  • Does it FTBFS currently?
  • Does it have a test suite? Make sure it's being run and will fail the build upon error.
  • Does it have a team bug subscriber? (poke bdmurray once a team signs on, so that he can update the master list in ubuntu-archive-tools)

  • Is the code translatable (if user visible)?
  • If it's a Python package, does it use dh_python?
  • If it's a Python package going on the desktop CD, will it pull in Python 2?

Packaging red flags

  • Does Ubuntu carry a delta?
  • If it's a library, does it either have a symbols file or use an empty argument to dh_makeshlibs -V? (pass such a patch on to Debian, but don't block on it). Note that for C++, see DailyRelease/FAQ for a method to demangle C++ symbols files.

  • Does it have a watch file?
  • Is its update history slow or sporadic?
  • Is the current release packaged?
  • Will entering main make it harder for the people currently keeping it up to date? (i.e. are they only MOTUs?)
  • Lintian warnings
  • Is debian/rules a mess? Ideally it uses dh7 and overrides to make it as tiny as possible.
  • Does debian/control use Built-Using. This may may indicate static linking which should be discouraged (excepting golang, see below)

  • If it's a statically compiled golang package:
    • Does the package use dh-golang (if not, suggest dh-make-golang to create the package)?
    • Does debian/control use Built-Using: ${misc:Built-Using} for each non'-dev' binary package (importantly, golang-*-dev packages only ship source files so don't need Built-Using)?

    • Does the package follow Debian Go packaging guidelines?

Upstream red flags

  • Errors/warnings during the build
  • Incautious use of malloc/sprintf
  • Uses of sudo, gksu, pkexec, or LD_LIBRARY_PATH
  • User nobody is strictly for NFS's use and must not be used by any running processes on the system.
  • Use of setuid needs evry careful design (prefer systemd to set those for services)
  • Important bugs (crashers, etc) in Debian or Ubuntu
  • Dependency on webkit, qtwebkit, seed or libgoa-*
  • Embedded source copies (this happens frequently with Go upstreams)
  • If this is a scope for the Unity Dash, does it honor the privacy settings?

Process states

This is supposed to be an overview of MIR bug status (as there was enough confusion in the past). Especially to make clear who's turn it is to act next on a MIR bug.

1. New/Confirmed (unassigned)

Bug is new for the MIR team to dispatch it to a MIR Team member

2. New/Confirmed (assigned)

on the TODO list of the assigned MIR team member

3. New/Confirmed (assigned to Security)

on the TODO list of the Security Team

4. In Progress (any)

MIR team ack (and if needed Security Team ack) done, but now needs the Dependency/Seed change to happen to pull it into Main

5. Fix Committed (any)

All of the above done, waiting for an AA to promote the packages

6. Fix Released (any)

Case resolved by an AA

7. Incomplete (any)

Questions/Requests were raised for the bug reporter to resolve/clarify

8. Won't Fix (any)

Final Nack or given up by the bug reporter

9. Invalid (any)

No Feedback for a while when in incomplete status

          Answered       +-7----------+     No feedback        +-9-------+
      +------------------+ Incomplete +------------------------> Invalid |
      |   old assign can | any        <--------------------+   | any     |
      |   be kept as-is  +-----^------+                    |   +---------+
      |                        |    question               |
      |                        |                           |
+-1---v-------+  Triage  +-2---+----+ Security needed  +-3-+------------------+
| New         +--------->+ New      +----------------->+ New                  |
| unassigned  |          | assigned |                  | assigned to Security |
+-------------+          +-+--+--+--+                  +-+--+--+--------------+
                           |  |  |    +-8-----------+    |  |  |
                           |  |  +----> Won't Fix   <----+  |  |
                           |  |   Nack| any         | Nack  |  |
                           |  |       +-------------+       |  |
                           |  |                             |  |
                           |  |       +-4-----------+       |  |
                           |  +-------> In Progress <-------+  |
                           |    Ack   | any         |  Ack     |
                           |          +-------------+          |
                           |             |Change that pulls    |
                           |             |into Main            |
+-6------------+           |         +-5-v-----------+         |
| Fix Released |           +---------> Fix Committed <---------+
| any          <---------------------+ any           |
+--------------+  AA promotes        +---------------+

Note1: Since many people set things to confirmed once they are sure it "exists" all too often bugs get set to confirmed. Since confirmed does not have any meaning for our process we will handle new<->confirmed as if the'y be the same.

Note2: All other combinations are undefined and should be fixed up to one of the defined states

Tools

  • check-mir can be run from a checked out source and tell you which dependencies are in universe.

  • seeded-in-ubuntu PACKAGE can tell you whether and how a given PACKAGE is seeded

  • reverse-depends can tell you reverse source or binary depends, per component

  • The component mismatch map

Making Life Easier for Archive Team Members

To help prevent promotion of packages that cause component mismatches, we can do two things:

  1. Run check-mir and make sure that all dependencies have a MIR. We don't want to be surprised by a dependency after a package is promoted.

  2. List all distinct binary packages that should be promoted. Often a source package will have binary packages that aren't actually needed in main. Things like -doc or -autopilot. These can stay in universe, and it is a kindness to list only the packages we need for the archive team member that does the promotion.

Bug Lists


CategoryUbuntuTeams

MIRTeam (last edited 2020-03-24 15:53:06 by paelzer)