Contents

  1. Kernel (regular)
  2. Kernel (emergency)
    1. Patching
    2. Building
    3. Testing the kernel
    4. Publishing
      1. ABI bump for -security and -updates pockets
      2. ABI bump for -security pocket only
  3. Signed updates
    1. Publishing to the Ubuntu archive
    2. Publishing to Ubuntu ESM
  4. Image-based updates
    1. Ubuntu Core
      1. Testing
        1. Series 16+
        2. 15.04
        3. Integrating into uvt
      2. Image generation (16 and higher)
        1. Using existing GA images
        2. Generating kvm images (16)
        3. Generating raspberry pi2 images (16)
        4. Generating dragonboard images (16)
        5. Generating beaglebone black images (16)
        6. Using external drive for writable partition
      3. Image generation (15.04)
        1. Generating kvm images (15.04)
        2. Generating beaglebone black images (15.04)
      4. Publishing (userspace)
        1. Triage
      5. Publishing (kernel)
    2. Touch
      1. Testing
        1. Flashing devices
        2. Generating emulator images
      2. Publishing (userspace)
        1. Triage
      3. Publishing (kernel)
  5. Mozilla
    1. Patching and Building
    2. Mozilla Pretesting Schedule
    3. Testing mozilla browsers
      1. Ubuntu 10.04 LTS and higher
      2. Notes on test-browser.py
    4. Testing mozilla thunderbird
    5. Mozilla Regressions
    6. Mozilla Publishing
  6. Chromium Browser
    1. [Old steps] Upload and Publication
    2. Sponsoring from Osomon updates
    3. Testing
  7. Konqueror browser
    1. Testing
  8. Rekonq browser
    1. Testing
  9. OpenJDK
    1. Package Preparation
      1. Package backports to stable releases
      2. Patch backports to stable releases
        1. OpenJDK 6
      3. New package generation
    2. Building
    3. Testing
    4. Distro Patching
  10. tzdata
  11. Update Manager
  12. Secure Boot
  13. Partner
  14. Sponsoring MariaDB Security Updates
  15. Sponsoring on private-fileshare.canonical.com
    1. Setup

This page describes the publication process for packages when it deviates from SecurityTeam/UpdateProcedures.

Kernel (regular)

Security updates are now a part of the regular kernel cadence. As such, the tracking and publication differs from other updates, and is detailed in SecurityTeam/UpdatePublication/Kernel.

Kernel (emergency)

Generally speaking, the kernel follows a different procedure for publication and tracking, which is detailed in SecurityTeam/UpdatePublication/Kernel. In case of an emergency update done outside of this process, the procedures below can be used.

Patching

Patching and testing the kernel is primarily the responsibility of the Ubuntu Kernel team who follow KernelTeam/KernelMaintenance. Tracking kernel CVEs, building patched kernels and publishing those kernels is the responsibility of the Ubuntu Security team. As such, the Ubuntu Security team should:

Building

Once the kernel team is satisfied with their patching and testing, they will provide packages on chinstrap.canonical.com, currently in chinstrap:~smb/security/srcpkg. Since the kernels can be large, the packages should be remotely signed and uploaded from chinstrap (see below). To verify, sign and upload:

  1. On chinstrap, copy the kernel team's packages to ~/sign: 
    $ cd ~smb/security/srcpkg/ # requires you are in the 'kernel_devs' group
$ test -d ~/sign/ || mkdir -m 0750 ~/sign/ ; chgrp ubuntu_security ~/sign/
$ cp * ~/sign/

  2. On local system (may require setup, see below), verify and sign the packages in ~/sign on chinstrap: 

    $ $UST/package-tools/u-verify-chinstrap      # verify the signatures in ~/sign
$ $UST/package-tools/u-sign-chinstrap        # sign the packages in ~/sign
$ $UST/package-tools/u-verify-chinstrap      # reverify the signatures in ~/sign

  3. If needed, on chinstrap setup the kern-up symlink: 

    $ test -e ~/bin/kern-up || ln -s /home/jamie/bin/kern-up ~/bin/kern-up

  4. On chinstrap, perform a test upload with kern-up. Eg: 

    $ cd ~/sign
$ ~/bin/kern-up # | sed 's/ security\-/ security-proposed-/' # for proposed ppa
dput security-dapper linux-source-2.6.15_2.6.15-55.87_source.changes
dput security-hardy linux_2.6.24-28.75_source.changes
dput security-jaunty linux_2.6.28-19.64_source.changes
dput security-karmic linux_2.6.31-22.63_source.changes
dput security-karmic linux-mvl-dove_2.6.31-214.30_source.changes
dput security-karmic linux-ec2_2.6.31-307.17_source.changes
dput security-lucid linux_2.6.32-24.41_source.changes
dput security-lucid linux-mvl-dove_2.6.32-208.24_source.changes
dput security-lucid linux-meta-mvl-dove_2.6.32.208.11_source.changes
dput security-lucid linux-ec2_2.6.32-308.15_source.changes
dput security-lucid linux-ti-omap_2.6.33-502.10_source.changes
dput security-lucid linux-fsl-imx51_2.6.31-608.19_source.changes

    Compare the output of kern-up with Kernel/Dev/ABIPackages. Ignore the netbook kernels cause they are outside the archive. Also, linux-qcm-msm/lucid is abandoned. If there is an ABI bump, then the ABI meta source package is also listed, otherwise it is not. Every "topic branch" (i.e. source package and referred to as 'git branch' in Kernel/Dev/ABIPackages) has up to two "meta" packages that define ABI, but normally there is just one. Sometimes there is an additional "ports" meta for the non-supported archs. Kernel/Dev/ABIPackages will always have the most up to date information, so consult it with each update (kern-up may need to be adjusted if the kernel team makes changes).

  5. Compare the ABIs of the packages output by kern-up with the archive. If there is an ABI bump and the meta package is missing, contact the kernel team.

  6. On chinstrap, upload the kernels (see the 'Setup' section below if publishing for the first time):

    • If non-ABI bump, do ~/bin/kern-up --real

    • If ABI bump:

      1. Take the output of ~/bin/kern-up and run the individual dput commands for each kernel and meta package, being careful to not upload any ABI-tracking packages at this time

      2. Wait for the kernels to build on all architectures

      3. After the kernels are finished building, for each of the remaining ABI-tracking packages (as seen in the output of ~/bin/kern-up), run the dput commands for that package

    • If destined for the ubuntu-security-proposed PPA, take the output of ~/bin/kern-up from above (after uncommenting the pipe to sed) and run the individual dput commands

  7. On chinstrap, verify all the packages were uploaded by comparing the number of .source.changes files with the number of .upload files in the ~/sign directory: 
    $ ls -1 ~sign/*_source.changes | wc -l
$ ls -1 ~sign/*.upload | wc -l

Testing the kernel

Most testing is performed by the kernel team. The Ubuntu Security should at a minimum do the following:

If there are reproducers or test cases, try to forward them to the kernel team (or better yet, integrate them into QRT before they do their testing). Private reproducers will need to be tested by the Ubuntu Security team. When possible, include a regression test for the patched functionality along with the test to see if the bug is fixed (ie, "Did this fix the bug? Did this introduce a regression?"). It is probably a good idea to adjust test_updated_modules() in $QRT/scripts/test-kernel-root-ops.py for any modules that have been updated (this will perform a modinfo, modprobe and rmmod on the module).

Finally, using virtualization for testing is fine most of the time, but if the patch is for a problem with real hardware, every effort should be made to test the patch on that hardware.

Publishing

In general, publication is the same as with other security updates. Keep in mind the following:

ABI bump for -security and -updates pockets

When a kernel is being built for -security that will introduce an ABI bump for both -security and -updates, the following items must be built in order:

When publishing, publish the kernel and ABI-tracked packages first, just to avoid any glitches where the meta package would get successfully published but something would block the kernel packages. Once the kernel package publications are verified in the archive, the meta package can be safely published.

Once the kernel is published in -security, it can be pocket-copied normally to -updates.

ABI bump for -security pocket only

When a -proposed kernel has an ABI bump and makes it into -updates, then the next security update kernel will be an ABI bump for -security only users (since security fixes pull from -updates). Since this is an ABI bump for -security only users, the ABI meta source packages and ABI-tracking source packages must be copied from -updates to -security after all of the -security kernels are mirrored. Look at Kernel/Dev/ABIPackages for the list of packages to copy over. For example, if we have a security update for the 'master' kernel (ie, not arm, not backports, etc) and lucid-security currently has 2.6.32-25.45, lucid-updates has 2.6.32-26.47 and the pending lucid-security update has 2.6.32-26.48, then:

Signed updates

From Ubuntu 14.04 forward, grub2 updates consist of two source packages: grub2 and grub2-signed. At a high level, publication requires:

In addition to grub2, the following packages use the same process:

Publishing to the Ubuntu archive

  1. Upload grub2 source to one of the security team's PPAs in the normal way
  2. Upload grub2-signed source with an updated Build-Depends from the grub2 upload in step '1' to one of the security team's PPAs in the normal way

  3. At publication, copy grub2 source and binaries to -proposed to generate the signed efi artifacts using the copy-package tool from ubuntu-archive-tools. Eg, for focal from the private security PPA:

    $ cd $UAT && ./copy-package --include-binaries --from-suite focal --from ~ubuntu-security/ubuntu/ppa --to ubuntu --to-suite focal-proposed --unembargo -y grub2

  4. Accept the package from the Unapproved queue into -proposed via the Ubuntu archive queue. Eg, for focal, https://launchpad.net/ubuntu/focal/+queue?queue_state=1&queue_text=

  5. Accepting the package into -proposed in the last step triggers Launchpad to perform signing and generate the 'signed efi artifacts'. When they show up in the Unapproved queue, they must be accepted before moving to the next step

  6. Copy grub2-signed source (ie, omit --include-binaries) to -proposed (to build the grub2-signed binaries with the signed efi artifacts) using the copy-package tool from ubuntu-archive-tools. Eg, for focal from the private security PPA (once in -proposed, the grub2-signed package will build for each architecture that the package supports. As part of the build process, the package will pull the signed efi artifacts from -proposed for grub2 and put them into the resulting binaries):

    $ cd $UAT && ./copy-package --from-suite focal --from ~ubuntu-security/ubuntu/ppa --to ubuntu --to-suite focal-proposed --unembargo -y grub2-signed

  7. Once the grub2-signed package is built (verify with https://launchpad.net/ubuntu/+source/grub2-signed/+publishinghistory) and after performing any additional testing, copy the grub2 and grub2-signed source and binaries in lockstep from -proposed to the -security pocket. Eg, for focal:

    $ cd $UAT && ./copy-package --from ubuntu --from-suite focal-proposed --to ubuntu --to-suite focal-security --unembargo --auto-approve -y grub2 grub2-signed

    Alternatively, simply use the sru-release tool. Eg:

    $ cd $UAT && ./sru-release --security <release name> grub2 grub2-signed

Publishing to Ubuntu ESM

Publishing to ESM requires the same process as the Ubuntu archive except a) instead of copying to -proposed for signing the packages are copied to the ESM signing ppa and b) instead of publishing to the -security pocket the packages are copied to an ESM infra ppa.

Note: normally ESM packages are built in an ESM staging PPA, tested and pushed from the ESM staging ppa to ESM proper. While we could build in ESM infra staging, copy to ESM signing and then push to ESM infra, we prefer for the signed packages to go to ESM infra staging for testing before pushing to ESM infra. As such, the below describes uploading first in the security private PPA (but any private ppa with only -security enabled will do), then copying to ESM signing, then pushing to ESM infra staging (for testing) and finally pushing to ESM infra.

  1. Upload grub2 source to one of the security team's non-ESM PPAs in the normal way
  2. Upload grub2-signed source with an updated Build-Depends from the grub2 upload in step '1' to one of the security team's non-ESM PPAs in the normal way

  3. At publication, copy grub2 source and binaries to the ESM signing ppa (https://launchpad.net/~canonical-signing/+archive/ubuntu/esm/+packages) using the copy-package tool from ubuntu-archive-tools. Eg, for trusty from the private security PPA (once in the signing PPA, the grub2 package will be shown with the 'gear icon'. As part of this process, efi signed artifacts are generated behind the scenes (ie, unlike the archive queue, the signed artifacts can't be seen via the normal ppa pages)):

    $ cd $UAT && ./copy-package --include-binaries --from ~ubuntu-security/ubuntu/ppa --from-suite trusty --to ~canonical-signing/ubuntu/esm --to-suite trusty --unembargo --auto-approve -y grub2

  4. When the grub2 packages from step 3 are published to the ESM signing PPA (checkmark icon), copy grub2-signed source (ie, omit --include-binaries) to the ESM signing PPA (to build the grub2-signed binaries with the signed efi artifacts) using the copy-package tool from ubuntu-archive-tools. Eg, for trusty from the private security PPA (once in the signing PPA, the grub2-signed package will build for each architecture that the package supports. As part of the build process, the package will pull the signed efi artifacts from the signing ppa for grub2 and put them into the resulting binaries):

    $ cd $UAT && ./copy-package --from ~ubuntu-security/ubuntu/ppa --from-suite trusty --to ~canonical-signing/ubuntu/esm --to-suite trusty --unembargo --auto-approve -y grub2-signed

  5. Once the grub2-signed package is built, copy the grub2 and grub2-signed source and binaries in lockstep from the ESM signing PPA to the ESM infra staging PPA. Eg, for trusty:

    $ cd $UAT && ./copy-package --include-binaries --from ~canonical-signing/ubuntu/esm --from-suite trusty --to ~ubuntu-esm/ubuntu/esm-infra-security-staging --to-suite trusty --unembargo --auto-approve -y grub2 grub2-signed

  6. Once verified and testing is complete, copy grub2 and grub2-signed from ESM infra staging to ESM infra in the normal way. Eg, for trusty:

    $ cd $UAT && ./copy-package --include-binaries --from ~ubuntu-esm/ubuntu/esm-infra-security-staging --from-suite trusty --to ~ubuntu-esm/ubuntu/esm-infra-security --to-suite trusty -y grub2 grub2-signed

Image-based updates

Ubuntu Touch, Ubuntu Core and (the future) Ubuntu Personal do not use apt for upgrades and instead get their updates via system-image updates (Ubuntu Touch and Ubuntu Core 15.04) or kernel/OS snap updates (Ubuntu Core and Personal in 16 and higher). In general, like with archive updates, it is sufficient to test the security update on the appropriate channel with kvm and add device testing as needed (eg, testing NetworkManager on a phone image or an arm-specific bug on a beaglebone black Ubuntu Core image).

Ubuntu Core

Testing

Testing can normally happen in a VM. Series 16, 18 and 20 images are available on cdimage (see below for custom VM/image generation). Start with:

Series 16+

See https://developer.ubuntu.com/core/get-started/kvm for details.

Some tips:

Most of these above will eventually be configurable via 'snap get core/snap set core'.

On early series 16 images netplan was used but it wasn't complete-- every reboot results in a new IP address even though the MAC didn't change. This shouldn't be an issue with newer images, but if it comes up it can be worked around with:

$ virsh net-edit default # add 'host' entries to dhcp section
...
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='52:54:00:52:ea:29' name='snappy-16-amd64' ip='192.168.122.64'/>
    </dhcp>
...
$ virsh net-destroy default
$ virsh net-start default

15.04

Some tips:

Integrating into uvt

Image generation (16 and higher)

Using existing GA images

By far the easiest way to get started is to use the official images. There are official images for amd64, i386, pi2 (armhf), pi3 (armhf) and dragonboard (arm64).

The steps are:

  1. Download an image and flash it to a device, boot it with kvm or integrate into uvt (see above)

  2. Start the image the first time, configure networking, etc
  3. Run 'sudo snap refresh' and reboot

  4. If you want another channel (eg, 'edge'): 

    $ sudo snap refresh --edge core
$ sudo snap refresh --edge pc-kernel
$ sudo snap refresh --edge pc

Generating kvm images (16)

  1. Install ubuntu-image with (currently need edge channel): 

    $ sudo snap install --channel=edge --devmode ubuntu-image

  2. Download the amd64 model assertion from http://people.canonical.com/~vorlon/official-models/

  3. Generate the image:

    • edge: 

      $ sudo ubuntu-image --image-size=8G -c edge -o snappy-amd64.img pc-amd64-model.assertion

    • beta: 

      $ sudo ubuntu-image --image-size=8G -c beta -o snappy-amd64.img pc-amd64-model.assertion

    • stable (emergency): 

      $ sudo ubuntu-image --image-size=8G -c stable -o snappy-amd64.img pc-amd64-model.assertion

See https://github.com/CanonicalLtd/snappy-docs/blob/master/core/images.md for more information.

http://cdimage.ubuntu.com/ubuntu-core/xenial/daily-preinstalled/current/ has pregenerated images.

Generating raspberry pi2 images (16)

Generating the image itself is the same as with 'Generating kvm images (16.04)' (above), except choose the pi2-model.assertion model assertion. Eg:

  1. Generate the image:

    $ sudo ubuntu-image -c beta -o snappy-pi2.img pi2-model.assertion

  2. flash it: sudo dd if=snappy-pi2.img of=/dev/sdX bs=32M && sync

Note: depending on your host hardware, the device may either be /dev/sdX or /dev/mmcblkX

Note 2: need to connect a keyboard and monitor for console-conf. It downloads your ssh key from LP for the email address you provide

Note 3: may want to use the 'godd' snap for flashing. Eg: cat snappy-pi2.img | sudo godd - /dev/sdX

Generating dragonboard images (16)

Generating the image itself is the same as with 'Generating kvm images (16.04)' (above), except choose the dragonboard-model.assertion model assertion. Eg:

  1. Generate the image:

    $ sudo ubuntu-image -c beta -o snappy-dragonboard.img dragonboard-model.assertion

  2. flash it: sudo dd if=snappy-dragonboard.img of=/dev/sdX bs=32M && sync

Note: depending on your host hardware, the device may either be /dev/sdX or /dev/mmcblkX

Note 2: make sure the SD card dipswitch is switched on for SD card booting

Note 3: may want to use the 'godd' snap for flashing. Eg: cat snappy-dragonboard.img | sudo godd - /dev/sdX

Note 4: need to connect a keyboard and monitor for console-conf. It downloads your ssh key from LP for the email address you provide

Generating beaglebone black images (16)

Beaglebone Black is a community kernel and gadget and is not officially supported. As such, you need to create your own model assertion and specify 'linux-generic-bbb' as the kernel snap and 'bbb' as the gadget snap. See https://docs.ubuntu.com/core/en/guides/build-device/board-enablement for details of creating a snap signing key and how to create the model assertion.

Assuming the model assertion is named 'my-bbb-model.assertion', then generating the image itself is the same as with 'Generating kvm images (16.04)' (above), except choose the my-bbb-model.assertion model assertion. Eg:

  1. Generate the image:

    $ sudo ubuntu-image -c edge -o snappy-bbb-stable.img my-bbb-model.assertion

  2. flash it: sudo dd if=snappy-bbb.img of=/dev/sdX bs=32M && sync

http://people.canonical.com/~ogra/snappy/all-snaps/daily/current/ has pregenereated images.

Note: you need an sdcard (eg 8G) instead of using internal storage at the time of this writing

Note 2: depending on your host hardware, the device may either be /dev/sdX or /dev/mmcblkX

Note 3: it is recommended to remove the eMMC bootloader with sudo dd if=/dev/zero of=/dev/mmcblk1 bs=1024 count=1024. You may want to make a backup of this (eg, sudo dd if=/dev/mmcblk1 of=./bbb.emmc bs=1024 count=1024). See the snappy-devel mailing list for details.

Note 4: may want to use the 'godd' snap for flashing. Eg: cat snappy-bbb.img | sudo godd - /dev/sdX

Note 5: unless using a serial console, need to connect a keyboard and monitor for console-conf. It downloads your ssh key from LP for the email address you provide

Note 6: since you can't install an unasserted kernel over an asserted one, when testing a new kernel, you might need to build an image and kernel so you can later perform locally unasserted kernel installs

Using external drive for writable partition

Sometimes it might be handy to boot off of the SD card but use an external USB hardrive for the 'writable' partition. From irc: 

#snappy: <@ogra> jdstrand, do you actually want to boot without SD ? (sorry, 
                 i'm at a trade show so only saw your ping now) ... if you can 
                 live with SD, just copy the writable partition over to the USB 
                 drive, re-label the SD one to "writable-old" and make sure the 
                 USB one is labeled "writable" ...

Eg, on a raspberry pi3, this procedure could be used:

  1. shutdown the device, remove the SD card and insert it into another computer to relabel it: Eg, assuming the SD shows up as /dev/mmcblk0... on your computer: 

    $ sudo umount /dev/mmcblk0p1  # unmount the automounted partitions
$ sudo umount /dev/mmcblk0p2
$ sudo e2label /dev/mmcblk0p2 # show current label
writable
$ sudo e2label /dev/mmcblk0p2 writable-old
$ sudo e2label /dev/mmcblk0p2
writable-old

  2. partition an external drive with a single partition using gdisk. Here is some output for the end results of an external disk (/dev/sda) that has been modified in this way: 

    $ sudo gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.4

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 3906963456 sectors, 1.8 TiB
Model: easystore 25FC  
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 5715C8FF-5530-490A-937D-5D1101095E0A
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 3906963422
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      3906961407   1.8 TiB     8300

$ sudo e2label /dev/sda1
writable
  3. copy over everything from writable-old (eg, /dev/mmcblk0p2 in the above) to writable (eg, /dev/sda1 in the above)

  4. put the SD card in the device and plug the USB drive into the device and boot. Then check df output to confirm writable is on the external drive: 

    $ df
...
/dev/sda1      1921770696 1445492 1822634836   1% /writable
...
/dev/mmcblk0p1     129039   45023      84016  35% /boot/uboot

Image generation (15.04)

This is gleaned from information found here:

Before you start with anything, you'll need to install some tools from the snappy-dev-tools PPA. See the getting started page for details.

You can see all the available images with (there may be additional historical channels in the output, but the below is what you should use):

$ ubuntu-device-flash query --list-channels --device=generic_amd64 | grep core
ubuntu-core/15.04/edge
ubuntu-core/15.04/stable
ubuntu-core/16/edge
ubuntu-core/16/stable
ubuntu-core/rolling/alpha
ubuntu-core/rolling/edge

Typically you'll use these channels for testing security updates:

Generating kvm images (15.04)

Useful options:

That's it (see 'Ubuntu Core/Testing' above).

Generating beaglebone black images (15.04)

Note: you need an sdcard (eg 8G) instead of using internal storage at the time of this writing

Note 2: depending on your host hardware, the device may either be /dev/sdX or /dev/mmcblkX

Note 3: it is recommended to remove the eMMC bootloader with sudo dd if=/dev/zero of=/dev/mmcblk1 bs=1024 count=1024. You may want to make a backup of this (eg, sudo dd if=/dev/mmcblk1 of=./bbb.emmc bs=1024 count=1024). See the snappy-devel mailing list for details.

Publishing (userspace)

In general, Ubuntu Core stable images/OS snaps are released on a cadence and therefore will bundle security updates every few weeks. See Note for cloud images, cloud archive, Ubuntu Core and/or Ubuntu Touch for details.

Triage

When packages are in the image PPA or the candidate images, the CVE for the affected release should be marked pending (<version>). Eg:

Candidate: CVE-2015-8317
References:
 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-8317
 http://www.ubuntu.com/usn/usn-2834-1
Description:
 The xmlParseXMLDecl function in parser.c in libxml2 before 2.9.3 allows
 context-dependent attackers to obtain sensitive information via an (1)
 unterminated encoding value or (2) incomplete XML declaration in XML data,
 which triggers an out-of-bounds heap read.
...
vivid_libxml2: released (2.9.2+dfsg1-3ubuntu0.2)
vivid/ubuntu-core_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
...

Notice that a USN was issues (2834-1) for vivid and vivid_libxml2 is marked released, however vivid/ubuntu-core_libxml2 is marked pending (2.9.2+dfsg1-3ubuntu0.2) because that version has not yet been provided in an OTA update for the stable images.

When a new OTA stable update is available, the triager should:

  1. verify in the image or OS snap manifest what was included (15.04 images can use dpkg -l (be sure to pipe to another program such as grep or redirect to a file to avoid truncation); 16 and higher has this information in /usr/share/snappy/dpkg.list for the OS snap and /snaps/canonical-*-linux.canonical/current/dpkg.list for the kernel snap). Eg, check if the manifest for the latest OS snap included libxml2 2.9.2+dfsg1-3ubuntu0.2

  2. If the manifest has the new release, see what CVEs it fixes with cd $UCT ; grep 'vivid/ubuntu-core_<srcpkg>: pending (' ./active/CVE*. Eg:

    $ grep 'vivid/stable-phone-overlay_libxml2: pending (' ./active/CVE*
./active/CVE-2015-7942:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8241:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8242:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8317:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)

  3. if and only if the image/OS snap has a fixed version, use $UCT/scripts/mass-cve-edit for any fixed CVEs. Eg: 

    $ cd $UCT ; ./scripts/mass-cve-edit -p libxml2 -r vivid/ubuntu-core -s released -v 2.9.2+dfsg1-3ubuntu0.2 CVE-2015-8317 CVE-2015-8242 CVE-2015-8241 CVE-2015-7942
CVE-2015-8317...
  vivid/ubuntu-core_libxml2 updated
CVE-2015-8242...
  vivid/ubuntu-core_libxml2 updated
CVE-2015-8241...
  vivid/ubuntu-core_libxml2 updated
CVE-2015-7942...
  vivid/ubuntu-core_libxml2 updated

Publishing (kernel)

In general, Ubuntu Core stable images/kernel snaps are released on a cadence and therefore will bundle security updates every few weeks. See Note for cloud images, cloud archive, Ubuntu Core and/or Ubuntu Touch for details.

Touch

Testing

The summary here was gleaned from:

You can see all the available images with (there may be additional historical channels in the output, but the below should be what you should use):

$ ubuntu-device-flash query --list-channels --device=mako | grep touch
ubuntu-touch/devel/ubuntu
ubuntu-touch/devel/ubuntu-developer
ubuntu-touch/devel-proposed/ubuntu-developer
ubuntu-touch/devel-proposed/ubuntu
ubuntu-touch/rc/ubuntu
ubuntu-touch/rc/ubuntu-developer
ubuntu-touch/rc-proposed/ubuntu
ubuntu-touch/rc-proposed/ubuntu-developer
ubuntu-touch/stable/ubuntu
ubuntu-touch/stable/ubuntu-developer
ubuntu-touch/stable-proposed/ubuntu
ubuntu-touch/stable-proposed/ubuntu-developer

Typically you'll use these channels for testing security updates:

Sideload click apps with:

$ pkcon install-local --allow-untrusted /path/to/click

Flashing devices

Generating emulator images

First install the necessary packages:

$ sudo apt-get install ubuntu-emulator ubuntu-emulator-runtime

Then create images:

A few potentially useful options:

You can run the emulator with:

$ ubuntu-emulator run stable.x86
$ ubuntu-emulator run rc-proposed.x86
$ ubuntu-emulator run --scale=0.75 devel-proposed.x86

Some things to keep in mind:

Publishing (userspace)

In general, Ubuntu Touch images are released on a cadence and therefore will bundle security updates every few weeks. See Note for cloud images, cloud archive, Ubuntu Core and/or Ubuntu Touch for details.

Ubuntu Touch images are built using the Ubuntu archive as a base with a PPA overlay for anything additional or updated to have in the image. The current Ubuntu Touch images are built with Ubuntu 15.04 and the vivid packages from the stable-phone-overlay. Security and SRU update to the Ubuntu archive for the base release will automatically flow into the daily rc-proposed images for the next OTA update if the updated package doesn't exist in the PPA. When the package does exist in the PPA, it will need to be updated using the standard silo process. The basic steps are (TODO: clean this up):

  1. build packages with stable-phone-overlay sources (both vivid (rc-proposed) and wily (devel-proposed))
  2. request silo in the direct upload way (document)
  3. upload to silo, wait for them to finish building
  4. do a 'watchonly' build step
  5. grab the packages and test in the emulator/on a device
  6. note testing performed in comment, edit the ticket to say 'QA requested'
  7. when they signoff, do 'Publish' and sign off on packaging changes

Triage

When packages are in the stable-phone-overlay the CVE for the affected release should be marked pending (<version>). Eg:

Candidate: CVE-2015-8317
References:
 http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2015-8317
 http://www.ubuntu.com/usn/usn-2834-1
Description:
 The xmlParseXMLDecl function in parser.c in libxml2 before 2.9.3 allows
 context-dependent attackers to obtain sensitive information via an (1)
 unterminated encoding value or (2) incomplete XML declaration in XML data,
 which triggers an out-of-bounds heap read.
...
vivid_libxml2: released (2.9.2+dfsg1-3ubuntu0.2)
vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
...

Notice that a USN was issues (2834-1) for vivid and vivid_libxml2 is marked released, however vivid/stable-phone-overlay_libxml2 is marked pending (2.9.2+dfsg1-3ubuntu0.2) because that version has not yet been provided in an OTA update for the stable images.

When a new OTA stable update is available, the triager should:

  1. update a device or emulator to the latest OTA. Eg:

    $ system-image-cli -i
current build number: 9
channel: ubuntu-touch/stable/meizu.en
version version: 9
version tag: OTA-9
...

  2. phablet-shell (or adb shell) in one terminal and search for the package with apt-cache policy <binpkg> or dpkg -l|grep <version> (be sure to always pipe dpkg -l output to another program such as grep or redirect to a file to avoid truncation). Eg:

    $ apt-cache policy libxml2
libxml2:
  Installed: 2.9.2+dfsg1-3ubuntu0.2
  Candidate: 2.9.2+dfsg1-3ubuntu0.2
  Version table:
 *** 2.9.2+dfsg1-3ubuntu0.2 0
        100 /var/lib/dpkg/status

  3. cd $UCT ; grep 'vivid/stable-phone-overlay_<srcpkg>: pending (' ./active/CVE* in one terminal. Eg:

    $ grep 'vivid/stable-phone-overlay_libxml2: pending (' ./active/CVE*
./active/CVE-2015-7942:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8241:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8242:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)
./active/CVE-2015-8317:vivid/stable-phone-overlay_libxml2: pending (2.9.2+dfsg1-3ubuntu0.2)

  4. if and only if the overlay has a fixed version, use $UCT/scripts/mass-cve-edit for any fixed CVEs. Eg: 

    $ cd $UCT ; ./scripts/mass-cve-edit -p libxml2 -r vivid/stable-phone-overlay -s released -v 2.9.2+dfsg1-3ubuntu0.2 CVE-2015-8317 CVE-2015-8242 CVE-2015-8241 CVE-2015-7942
CVE-2015-8317...
  vivid/stable-phone-overlay_libxml2 updated
CVE-2015-8242...
  vivid/stable-phone-overlay_libxml2 updated
CVE-2015-8241...
  vivid/stable-phone-overlay_libxml2 updated
CVE-2015-7942...
  vivid/stable-phone-overlay_libxml2 updated

Publishing (kernel)

In general, Ubuntu Touch images are released on a cadence and therefore will bundle security updates every few weeks. See Note for cloud images, cloud archive, Ubuntu Core and/or Ubuntu Touch for details.

Ubuntu Touch kernel images are vendor kernels and not built from the Ubuntu archive. In general, the Ubuntu Security and Kernel teams will track and file bugs for these product kernel and alert the responsible internal Canonical team to update the product kernels. See the UCT documentation for more information on working with product kernels.

Mozilla

Patching and Building

Patching and building is currently the responsibility of the ubuntu-mozillateam, specifically ChrisCoulson as a backup. Mozilla products have a standing MicroReleaseException and so the ubuntu-mozillateam will get official tarballs from upstream, add/update the debian/ directory and push to the ubuntu-mozilla-security PPA. The ubuntu-mozillateam will also ask for USN to put in the changelog prior to preparing updates since upstream does not make public security vulnerabilities prior to release. People reading the changelog are then able to see the USN and look up the details online. Once packages are built, you can use the standard Ubuntu Security team's tools for publication (using the --ppa=ubuntu-mozilla-security option where appropriate).

The tarballs are built using a script in the packaging branches. To run the script, look in the upstream Mozilla tree to find the build tag of interest and then invoke debian/rules accordingly:

$ debian/rules get-orig-source DEBIAN_TAG=FIREFOX_42_0_BUILD2

The packaging branches can be found under the ~mozillateam's Launchpad code page:

There are corresponding branches for thunderbird.

Once the tarball is created, build the first package:

$ bzr-buildpackage -S -sa

For the remaining packages, use:

$ bzr-buildpackage -S -sd

Upstream Mozilla does not give us access to their security bugs, and we get all of your information from upstream's MFSAs (Mozilla Foundation Security Announcements). As such, when the update does not fix a security issue (have an assigned CVE) the developer that uploads packages to the ubuntu-mozilla-security PPA will file a placeholder bug to be used in the changelog. At time of publication, this placeholder bug is filled in with a link to the USN. As such, changelog entries are shorted and typically consist of something like:

firefox (14.0.1+build1-0ubuntu0.12.04.1) precise-security; urgency=low

  * New upstream stable release (FIREFOX_14_0_1_BUILD1)
    - see LP: #1024562 for USN information

Firefox and thunderbird, while they may share some of the same CVEs, will have separate '-1' preallocated USNs.

Mozilla Pretesting Schedule

Pretesting of Firefox and Thunderbird is performed on the Aurora (pre-beta) and Beta channels. The schedule below starts on the Mozilla release week.

Week

Day

Channel

Release

Arch

Notes

1

Monday

Release

All

Both

Coordinate with ChrisCoulson to guarantee packages will be ready

1

Tuesday

Release Day

1

Friday

Beta

Natty

i386

2

Tuesday

Aurora

Lucid

amd64

2

Friday

Beta

Oneiric

i386

3

Tuesday

Aurora

Oneiric

amd64

4

Tuesday

Beta

Natty

amd64

4

Friday

Aurora

Precise

i386

5

Tuesday

Beta

Precise

amd64

6

Tuesday

Beta

Lucid

i386

The Launchpad PPAs corresponding to the various Mozilla channels are listed below:

Mozilla Channel

Ubuntu Source Pkg

PPA

Release

Firefox

ubuntu-mozilla-security/ppa

Release

Thunderbird

ubuntu-mozilla-security/ppa

Beta

Firefox

mozillateam/firefox-next

Beta

Thunderbird

mozillateam/thunderbird-next

Aurora

Firefox

ubuntu-mozilla-daily/firefox-aurora

Aurora

Thunderbird

ubuntu-mozilla-daily/thunderbird-aurora

Testing mozilla browsers

Verify the QRT test-browser.py script for the all affected products passes for both i386 and amd64. This script will test a variety of functions, test pages, SSL, javascript, plugins, etc. Since some packages other than Firefox use XUL or NSS, the following gives information on basic testing procedures when a particular source package is updated. These instructions should provide good enough test coverage for a particular update, but are not intended to provide exhaustive testing procedures. To know which packages to test, look in the ubuntu-mozilla-security PPA at the source packages to be tested, and look up the test procedure in the tables below for that release. Eg: 

$ for i in firefox firefox-3.0 firefox-3.5 xulrunner-1.9.2 ; do copy_sppa_to_repos --ppa=ubuntu-mozilla-security $i ; done

It is recommended that your testing environment use the security team's vm-tools. Assuming your testing environment is properly setup, firefox testing with test-browser.py should not take more than 30 minutes per arch/release (eg, 4 hours total for 4 stable releases on both amd64 and i386, typically less).

Ubuntu 10.04 LTS and higher

Updated Source Package

Additional Affected Binaries

Testing procedure

firefox

N/A

sudo aa-enforce /etc/apparmor.d/usr.bin.firefox1 && $QRT/scripts/test-browser.py -v2

thunderbird

N/A

see $QRT/notes_testing/nss/README and $QRT/notes_testing/thunderbird for email, then also test addressbook and feed reader

  1. be sure there are no new AppArmor denials after running the script

  2. this will test java, flash, totem, etc so those applications don't have to be tested separately, but be sure to test both icedtea6-plugin and sun-java6-plugin. Can test the java plugin alone with $QRT/scripts/test-browser.py -v -t java

Notes on test-browser.py

The test-browser.py script is an interactive script that guides the tester through a number of actions that when completed should demonstrate that the browser is functional for the most important use cases. Unlike other QRT, it has a certain 'feel' to it, and if you have not used it before you should run the tests against the current version of the browser in the archive, then compare that to a test run against the version to be in the security update. Some things to keep in mind:

Testing mozilla thunderbird

There is not a QRT script for thunderbird at this point, however there are notes in QRT for how to test thunderbird. Specifically:

It is recommended that your testing environment use the security team's vm-tools. Assuming your testing environment is properly setup, thunderbird testing should not take more than 60 minutes per arch/release (Eg, 8 hours total for 4 stable releases on both amd64 and i386, typically less).

Mozilla Regressions

If during testing you find a regression in a Mozilla product, follow this procedure to alert the Ubuntu Desktop team:

Mozilla Publishing

The publication procedure is essentially the same as for regular security updates except:

Chromium Browser

Can check for new chromium releases here: https://omahaproxy.appspot.com/

[Old steps] Upload and Publication

chromium packages are currently supported by Chad Miller (chad). These packages are large, so sponsoring should be:

  1. On your local machine, get the new sources with: wget .../ (don't forget the trailing '/'!)

  2. On your local machine, compare the new sources with the the last published sources and verify the new packages follow MicroReleaseExceptions

  3. Once the packages are verified as ok, copy them to the ~/sign directory on chinstrap (you can wget --mirror -np these from chinstrap)

  4. Verify and sign the packages from you local machine (may require setup, see below):

    $ $UST/package-tools/u-verify-chinstrap      # verify the signatures in ~/sign
$ $UST/package-tools/u-sign-chinstrap        # sign the packages in ~/sign
$ $UST/package-tools/u-verify-chinstrap      # reverify the signatures in ~/sign

  5. dput from chinstrap (may require setup, see below):

    $ dput security-proposed-lucid chromium-browser_*.10.04.1_source.changes
$ dput security-proposed-maverick chromium-browser_*.10.10.1_source.changes

Rather than uploading directly to the security PPA, we instead basically use the Ubuntu Security team's sponsored upload procedures:

Sponsoring from Osomon updates

Chromium browser is update/build in canonical-chromium-buils

In order to sponsor it follow the steps:

  1. Copy chromium-browser to your local repo to test it:
    • e.g for a specific release: $UST/repo-tools/copy_sppa_to_repos --ppa=canonical-chromium-builds/stage chromium-browser --release=bionic

  2. Test it using this chromium test plans and the following testing section using $QRT.

  3. After tested, unembargo it. The -n option is used here to first check if the copy will be ok, after check type it without the -n option:
    • e.g: $UQT/security-tools/unembargo -n --ppa=canonical-chromium-builds/stage --release=bionic chromium-browser

  4. Then continue with the standard procedures to announce the update

Testing

As mentioned above, use https://git.launchpad.net/qa-regression-testing:/scripts/test-browser.py and document the results in the bug. For best results:

IMPORTANT: be sure to ask #webapps to test the new packages before publication for 12.10 and later

Konqueror browser

Testing

Use https://git.launchpad.net/qa-regression-testing:/scripts/test-browser.py. For best results:

Rekonq browser

Testing

Use https://git.launchpad.net/qa-regression-testing:/scripts/test-browser.py. For best results:

OpenJDK

Package Preparation

The typical approach for OpenJDK updates is:

The points of contact on Ubuntu Foundations are(as of 2015/01/27) doko (falling back to slangasek).

Package backports to stable releases

If have the package from the development release or Debian, you will need to update the debian/control file. This file is regenerated by using debian/rules within an i386 schroot with full build-depends installed. For example, for openjdk-7 (substitute 'openjdk-6' when preparing packages for it):

$ dpkg-source -x ./openjdk-7*.dsc # unpack the new package
$ cd ./openjdk-7*/
$ dch -i # adjust for stable release as appropriate
$ schroot -c <release>-i386 -u root
(<release>-i386)# apt-get install lsb-release
(<release>-i386)# apt-get build-dep openjdk-7
(<release>-i386)# su <your username>
(<release>-i386)$ touch debian/rules && debian/rules debian/control
...
debian/control did change, please restart the build
make: *** [debian/control] Error 1
(<release>-i386)$ exit
(<release>-i386)# exit

Then proceed to build the package as usual. Things to watch out for:

IMPORTANT: be sure to verify the following:

The following are listed for historical reference in case something gets dropped and to see how to deal with various issues. They should no longer be needed as of 2015/01/27:

Patch backports to stable releases

While using new upstream releases with package backports is preferred, sometimes backporting an isolated patch is needed.

OpenJDK 6

Upstream Vcs can be found here:

USN-2033-1 cherrypicked patches from trunk and 1.11 because the 1.12 branch wasn't updated yet. Due to how IcedTea patches work, the patches could not be applied as simple distribution patches. Following upstream patching, the Makefile.* files must be modified before running the IcedTea configure step. As such, USN-2033-1 adjusted debian/rules to apply debian/patches/ubuntu-security-NNNN-* before running configure:

As such, future updates can follow this procedure. On the latest stable release (or devel release):

$ debian/rules ubuntu-security # applies all the ubuntu-security-* patches
$ cd .. ; cp -a openjdk-6-6b27-1.12.6 openjdk-6-6b27-1.12.6.orig
<apply patches>
$ diff -Naurp openjdk-6-6b27-1.12.6.orig openjdk-6-6b27-1.12.6 > /tmp/ubuntu-security-NNNN-....patch
$ rm -rf openjdk-6-6b27-1.12.6
$ cp -a openjdk-6-6b27-1.12.6.orig openjdk-6-6b27-1.12.6
$ cp /tmp/ubuntu-security-NNNN-....patch debian/patches
$ fakeroot debian/rules patch && cd build && make patch # if this works, add to package and try to build it

The above may fail and you might have to adjust the following as needed:

If Makefile.am changes (which it most certainly will), need to run autotools to update Makefile.in. icedtea 1.13 needs automake 1.14, but this only exists in trusty and later, so on that release, use 'fakeroot debian/rules patch && cd build && make patch', then create a patch to update ubuntu-security-9999-Makefile.in.patch

New package generation

The Foundations team normally handles generating new upstream versions so this isn't normally needed (and may be out of date).

For openjdk updates, generally we take the icedtea tarballs that Andrew Hughes produces. These will also be announced to the openjdk distro-pkg-dev list. To incorporate them, do the following:

Building

If you receive a orig.tar.gz from the Foundations team for you to apply to a security update (as opposed to a full source package), be sure that you run debian/update-shasum.sh, apply debian/patches/icedtea-patch.diff, run sh autogen.sh then rm -rf autom4te.cache before building. Check debian/generate-debian-orig.sh to make sure you've done everythng required before building.

When building openjdk locally with UMT, depending on your build system's hardware configuration, you'll likely need to pass -C pkgbuild_ulimit_v="5242880" to umt build.

IMPORTANT: for some reason armel builds on lucid and oneiric sometimes will FTBFS with random segmentation faults (search for 'Segmentation' in the build log). If this happens, retry the build. Some armel builders don't seem to be able to handle openjdk well.

Testing

Review QRT/notes_testing/openjdk-6/README (it contains information on both openjdk-6 and openjdk-7). Basically, use test-openjdk.py --jdk=openjdk-[67] -v from QRT as well as QRT/notes_testing/openjdk-6/extract-test-results.sh. test-openjdk.py will test eclipse and netbeans as well as the web plugin on firefox and chromium-browser. extract-test-results.sh helps compare test suite runs in LP build logs. Running tests from the testsuite is documented in QRT/build_testing/openjdk/README.txt.

If find issues, use the upstream bug tracker.

Distro Patching

To patch openjdk, add patches to debian/patches, adjust the file paths in the patch, then add the patch to debian/rules. IcedTea has a couple of different targets:

tzdata

tzdata update mostly and in general is done by upgrading to new versions of orig.tar.gz upstream. There are two ways to do that:

Update Manager

Update Manager updates have some special post-publication steps that need to get done by an Archive Admin. See special instructions here.

Secure Boot

Please see the special instructions for performing and testing Secure Boot key databases.

Partner

After uploading to partner, an archive admin must process the upload.

Sponsoring MariaDB Security Updates

The MariaDB packages in Ubuntu receive consistent security support from Otto Kekäläinen. Otto provides the MariaDB security updates to the Ubuntu Security Team in the form a git tree that he maintains for use with git-buildpackage. It is easy to take Otto's changes and consume them into the typical umt-based workflow.

The example instructions below will focus on mariadb-10.0 packages in Ubuntu 16.04 LTS. Adjust the Ubuntu release and MariaDB package name accordingly.

Sponsoring on private-fileshare.canonical.com

Sometimes developers or teams will provide packages on private-fileshare.canonical.com. When these source packages are large, the packages can be remotely signed and uploaded from private-fileshare (you do not need nor should have your ~/.gnupg on private-fileshare!). The basic process is:

  1. If it doesn't already exist, create the ~/sign directory on private-fileshare with:

    $ test -d ~/sign/ || mkdir -m 0750 ~/sign/
$ chgrp ubuntu_security ~/sign/

  2. Copy the packages to sponsor into the ~/sign directory

  3. On local system (may require setup, see below), verify and sign the packages in ~/sign on private-fileshare: 
    $ $UST/package-tools/u-verify-private-fileshare      # verify the signatures in ~/sign
$ $UST/package-tools/u-sign-private-fileshare        # sign the packages in ~/sign
$ $UST/package-tools/u-verify-private-fileshare      # reverify the signatures in ~/sign
  4. dput the packages from private-fileshare (may require setup, see below)

Setup

SecurityTeam/PublicationNotes (last edited 2023-04-13 16:58:52 by leosilvab)