PublicationNotes

Differences between revisions 1 and 2
Revision 1 as of 2010-08-20 19:09:02
Size: 46
Editor: pool-71-123-2-34
Comment:
Revision 2 as of 2010-08-20 23:02:04
Size: 8133
Editor: pool-71-123-2-34
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
Describe SecurityTeam/PublicationNotes here. This page describes the publication process for packages when it deviates from SecurityTeam/UpdateProcedures.

== Kernel ==

=== Patching ===
Patching and testing the kernel is primarily the responsibility of the Ubuntu Kernel team. Tracking kernel CVEs, building patched kernel and publishing those kernels is the responsibily of the Ubuntu Security team. As such, the Ubuntu Security team should:
 * enter kernel CVEs into the Ubuntu CVE Tracker
 * forward this information to the kernel team
 * coordinate the timing of kernel security updates (usually monthly, unless a high priority CVE warrants an earlier date
 * coordinate the Ubuntu Kernel team's work with other vendors as appropriate

=== Building ===
Once the kernel team is satisfied with their patching and testing, they will provide packages on ```chinstrap.canonical.com```, currently in ```chinstrap:~smb/security/srcpkg```. Since the kernels are large, the packages should be remotely signed and uploaded from chinstrap (you do not need nor should have your ~/.gnupg on chinstrap!). To verify, sign and upload:
 0. on chinstrap:
 {{{
$ ~smb/security/srcpkg/
$ test -d ~/sign/ || mkdir -m 0750 ~/sign/ ; chgrp ubuntu_security ~/sign/
$ cp * ~/sign/
}}}
 0. on local system (may require setup, see below):
 {{{
$ $UST/package-tools/u-verify-chinstrap # verify the signatures in ~/sign on chinstrap
$ $UST/package-tools/u-sign-chinstrap # sign the packages in ~/sign on chinstrap
$ $UST/package-tools/u-verify-chinstrap # verify the signatures in ~/sign on chinstrap (to make sure u-sign-chinstrap didn't make an error)
}}}
 0. If needed, on chinstrap setup the ```kern-up``` symlink:
 {{{
$ test -f ~/bin/kern-up && ln -s /home/kees/bin/kern-up ~/bin/kern-up
}}}
 0. On chinstrap, perform a test upload with ```kern-up```. Eg:
 {{{
$ ~/bin/kern-up # test run
dput security-dapper linux-source-2.6.15_2.6.15-55.87_source.changes
dput security-hardy linux_2.6.24-28.75_source.changes
dput security-jaunty linux_2.6.28-19.64_source.changes
dput security-karmic linux_2.6.31-22.63_source.changes
dput security-karmic linux-mvl-dove_2.6.31-214.30_source.changes
dput security-karmic linux-ec2_2.6.31-307.17_source.changes
dput security-lucid linux_2.6.32-24.41_source.changes
dput security-lucid linux-mvl-dove_2.6.32-208.24_source.changes
dput security-lucid linux-meta-mvl-dove_2.6.32.208.11_source.changes
dput security-lucid linux-ec2_2.6.32-308.15_source.changes
dput security-lucid linux-ti-omap_2.6.33-502.10_source.changes
dput security-lucid linux-fsl-imx51_2.6.31-608.19_source.changes
}}}
 Compare the output of ```kern-up``` with Kernel/Dev/ABIPackages. Ignore the netbook kernels cause they are outside the archive. Also, linux-qcm-msm/lucid is abandoned. If there is an ABI bump, then the ABI meta source package is also listed, otherwise it is not. Every "topic branch" (i.e. source package and referred to as 'git branch' in Kernel/Dev/ABIPackages) has up to two "meta" packages that define ABI, but normally there is just one. Sometimes there is an additional "ports" meta for the non-supported archs. Kernel/Dev/ABIPackages will always have the most up to date information, so consult it with each update (```kern-up``` may need to be adjusted if the kernel team makes changes).
 0. Compare the ABIs of the packages output by ```kern-up``` with the archive. If there is an ABI bump and the meta package is missing, contact the kernel team. If there is an ABI bump, upload and build all the kernels and meta-packages first, then upload the ABI-tracking packages after they are built.
 0. On chinstrap, upload the kernels:
  * If non-ABI bump, do ```~/bin/kern-up --real```
  * If ABI bump:
   0. Take the output of ```~/bin/kern-up``` and run the individual dput command for each kernel and meta package, being careful to not upload any ABI-tracking packages at this time
   0. Wait for the kernels to build on all architectures
   0. For each of the remaining ABI-tracking packages (as seen in the output of ```~/bin/kern-up```), run the dput commands for that package
 0. on chinstrap, verify all the packages were uploaded by comparing the number of .source.changes files with the number of .upload files in the ~/sign directory:
 {{{
$ ls -1 ~sign/*_source.changes | wc -l
$ ls -1 ~sign/*.upload | wc -l
}}}

==== Setup ====
 * Depending on how your ssh_config is setup on your local machine, ```u-sign-chinstrap``` and ```u-verify-chinstrap``` may not work immediately. However, you can create these symlinks and it should work ok:
 {{{
$ ln -s $UST/package-tools/u-sign-chinstrap ~/bin/u-sign-chinstrap.canonical.com
$ ln -s $UST/package-tools/u-verify-chinstrap ~/bin/u-verify-chinstrap.canonical.com
}}}
 * On chinstrap, setup your ~/.dput.cf file as in SecurityTeam/UpdateProcedures. Eg:
 {{{
[security-lucid]
fqdn = ppa.launchpad.net
incoming = ~ubuntu-security/ubuntu/lucid
login = anonymous

[security-karmic]
fqdn = ppa.launchpad.net
incoming = ~ubuntu-security/ubuntu/karmic
login = anonymous

[security-jaunty]
fqdn = ppa.launchpad.net
incoming = ~ubuntu-security/ubuntu/jaunty
login = anonymous

[security-hardy]
fqdn = ppa.launchpad.net
incoming = ~ubuntu-security/ubuntu/hardy
login = anonymous

[security-dapper]
fqdn = ppa.launchpad.net
incoming = ~ubuntu-security/ubuntu/dapper
login = anonymous
}}}

=== Testing ===
Most testing is performed by the kernel team. The Ubuntu Security at a minimum should do the following:
 * using ```copy_sppa_to_repos``` from UST, copy the kernels to your local repository. There are instructions at the top of ```copy_sppa_to_repos``` for different kernels, ABI-tracking packages and meta-packages
 * perform upgrade testing for all affected releases for both i386 and amd64 (using the meta packages. This can be done by ensuring linux-image-generic (linux-image-amd64-generic or linux-image-386 for Dapper) is installed, then performing an ```apt-get dist-upgrade``` to pull in the packages from you local repository.
 * after upgrading, verify the following:
 {{{
$ uname -a
$ cat /proc/version_signature # for non-Dapper
}}}
 * verify lp:qa-regression-testing (QRT) test scripts for the kernel pass for both i386 and amd64. Run all $QRT/scripts/test-kernel*py scripts except test-kernel-hardening.py.

As a convenience, $QRT/notes_testing/kernel/kernel-test-wrapper.sh can be used to automate the above.

If there are reproducers or test cases, try to forward to the kernel team (or better yet, integrate them into QRT before they do their testing). Private reproducers will need to be tested by the Ubuntu Security team. Always try to include a regression test for the patched functionality along with the test to see if the bug is fixed (ie, "Did this fix the bug? Did this introduce a regression?).

Finally, using virtualization for testing is fine most of the time, but if the patch is for a problem with real hardware, every effort should be made to test the patch on that hardware.

=== Publishing ===
In general, publication is the same as with other security updates. Keep in mind the following:
 * Unembargo all non-meta packages at the same time, then after they are mirrored to security.ubuntu.com, upload the meta packages. This will ensure that people don't get a meta package that depends on a kernel that doesn't exist.
 * While not required, you can use the ```pull-usn-desc.py``` tool from UCT. This is helpful since kernel updates typically have many CVEs to describe in the USN. You give it the CVE list that is in new-usn.sh and it will output example text that you can paste into new-usn.sh and edit. Eg:
 {{{
$ cd $UCT
$ $UCT/scripts/pull-usn-desc.py --cve CVE-... --cve CVE-...
}}}
 * The title for the USN should be 'Linux kernel vulnerabilities'
 * The summary for the USN should be something like 'linux, linux-{ec2,fsl-imx51,mvl-dove,source-2.6.15,ti-omap} vulnerabilities'
 * In the minimum binaries section of the USN, use linux-image* (but no debug or dbgsym packages)

== Mozilla ==
TODO

== Chromium Browser ==
TODO

This page describes the publication process for packages when it deviates from SecurityTeam/UpdateProcedures.

Kernel

Patching

Patching and testing the kernel is primarily the responsibility of the Ubuntu Kernel team. Tracking kernel CVEs, building patched kernel and publishing those kernels is the responsibily of the Ubuntu Security team. As such, the Ubuntu Security team should:

  • enter kernel CVEs into the Ubuntu CVE Tracker
  • forward this information to the kernel team
  • coordinate the timing of kernel security updates (usually monthly, unless a high priority CVE warrants an earlier date
  • coordinate the Ubuntu Kernel team's work with other vendors as appropriate

Building

Once the kernel team is satisfied with their patching and testing, they will provide packages on chinstrap.canonical.com, currently in chinstrap:~smb/security/srcpkg. Since the kernels are large, the packages should be remotely signed and uploaded from chinstrap (you do not need nor should have your ~/.gnupg on chinstrap!). To verify, sign and upload:

  1. on chinstrap:
    $ ~smb/security/srcpkg/
    $ test -d ~/sign/ || mkdir -m 0750 ~/sign/ ; chgrp ubuntu_security ~/sign/
    $ cp * ~/sign/
  2. on local system (may require setup, see below):
    $ $UST/package-tools/u-verify-chinstrap      # verify the signatures in ~/sign on chinstrap
    $ $UST/package-tools/u-sign-chinstrap        # sign the packages in ~/sign on chinstrap
    $ $UST/package-tools/u-verify-chinstrap      # verify the signatures in ~/sign on chinstrap (to make sure u-sign-chinstrap didn't make an error)
  3. If needed, on chinstrap setup the kern-up symlink:

    $ test -f ~/bin/kern-up && ln -s /home/kees/bin/kern-up ~/bin/kern-up
  4. On chinstrap, perform a test upload with kern-up. Eg:

    $ ~/bin/kern-up # test run
    dput security-dapper linux-source-2.6.15_2.6.15-55.87_source.changes
    dput security-hardy linux_2.6.24-28.75_source.changes
    dput security-jaunty linux_2.6.28-19.64_source.changes
    dput security-karmic linux_2.6.31-22.63_source.changes
    dput security-karmic linux-mvl-dove_2.6.31-214.30_source.changes
    dput security-karmic linux-ec2_2.6.31-307.17_source.changes
    dput security-lucid linux_2.6.32-24.41_source.changes
    dput security-lucid linux-mvl-dove_2.6.32-208.24_source.changes
    dput security-lucid linux-meta-mvl-dove_2.6.32.208.11_source.changes
    dput security-lucid linux-ec2_2.6.32-308.15_source.changes
    dput security-lucid linux-ti-omap_2.6.33-502.10_source.changes
    dput security-lucid linux-fsl-imx51_2.6.31-608.19_source.changes

    Compare the output of kern-up with Kernel/Dev/ABIPackages. Ignore the netbook kernels cause they are outside the archive. Also, linux-qcm-msm/lucid is abandoned. If there is an ABI bump, then the ABI meta source package is also listed, otherwise it is not. Every "topic branch" (i.e. source package and referred to as 'git branch' in Kernel/Dev/ABIPackages) has up to two "meta" packages that define ABI, but normally there is just one. Sometimes there is an additional "ports" meta for the non-supported archs. Kernel/Dev/ABIPackages will always have the most up to date information, so consult it with each update (kern-up may need to be adjusted if the kernel team makes changes).

  5. Compare the ABIs of the packages output by kern-up with the archive. If there is an ABI bump and the meta package is missing, contact the kernel team. If there is an ABI bump, upload and build all the kernels and meta-packages first, then upload the ABI-tracking packages after they are built.

  6. On chinstrap, upload the kernels:
    • If non-ABI bump, do ~/bin/kern-up --real

    • If ABI bump:
      1. Take the output of ~/bin/kern-up and run the individual dput command for each kernel and meta package, being careful to not upload any ABI-tracking packages at this time

      2. Wait for the kernels to build on all architectures
      3. For each of the remaining ABI-tracking packages (as seen in the output of ~/bin/kern-up), run the dput commands for that package

  7. on chinstrap, verify all the packages were uploaded by comparing the number of .source.changes files with the number of .upload files in the ~/sign directory:
    $ ls -1 ~sign/*_source.changes | wc -l
    $ ls -1 ~sign/*.upload | wc -l

Setup

  • Depending on how your ssh_config is setup on your local machine, u-sign-chinstrap and u-verify-chinstrap may not work immediately. However, you can create these symlinks and it should work ok:

    $ ln -s $UST/package-tools/u-sign-chinstrap ~/bin/u-sign-chinstrap.canonical.com
    $ ln -s $UST/package-tools/u-verify-chinstrap ~/bin/u-verify-chinstrap.canonical.com
  • On chinstrap, setup your ~/.dput.cf file as in SecurityTeam/UpdateProcedures. Eg:

    [security-lucid]
    fqdn = ppa.launchpad.net
    incoming = ~ubuntu-security/ubuntu/lucid
    login = anonymous
    
    [security-karmic]
    fqdn = ppa.launchpad.net
    incoming = ~ubuntu-security/ubuntu/karmic
    login = anonymous
    
    [security-jaunty]
    fqdn = ppa.launchpad.net
    incoming = ~ubuntu-security/ubuntu/jaunty
    login = anonymous
    
    [security-hardy]
    fqdn = ppa.launchpad.net
    incoming = ~ubuntu-security/ubuntu/hardy
    login = anonymous
    
    [security-dapper]
    fqdn = ppa.launchpad.net
    incoming = ~ubuntu-security/ubuntu/dapper
    login = anonymous

Testing

Most testing is performed by the kernel team. The Ubuntu Security at a minimum should do the following:

  • using copy_sppa_to_repos from UST, copy the kernels to your local repository. There are instructions at the top of copy_sppa_to_repos for different kernels, ABI-tracking packages and meta-packages

  • perform upgrade testing for all affected releases for both i386 and amd64 (using the meta packages. This can be done by ensuring linux-image-generic (linux-image-amd64-generic or linux-image-386 for Dapper) is installed, then performing an apt-get dist-upgrade to pull in the packages from you local repository.

  • after upgrading, verify the following:
    $ uname -a
    $ cat /proc/version_signature    # for non-Dapper
  • verify lp:qa-regression-testing (QRT) test scripts for the kernel pass for both i386 and amd64. Run all $QRT/scripts/test-kernel*py scripts except test-kernel-hardening.py.

As a convenience, $QRT/notes_testing/kernel/kernel-test-wrapper.sh can be used to automate the above.

If there are reproducers or test cases, try to forward to the kernel team (or better yet, integrate them into QRT before they do their testing). Private reproducers will need to be tested by the Ubuntu Security team. Always try to include a regression test for the patched functionality along with the test to see if the bug is fixed (ie, "Did this fix the bug? Did this introduce a regression?).

Finally, using virtualization for testing is fine most of the time, but if the patch is for a problem with real hardware, every effort should be made to test the patch on that hardware.

Publishing

In general, publication is the same as with other security updates. Keep in mind the following:

  • Unembargo all non-meta packages at the same time, then after they are mirrored to security.ubuntu.com, upload the meta packages. This will ensure that people don't get a meta package that depends on a kernel that doesn't exist.
  • While not required, you can use the pull-usn-desc.py tool from UCT. This is helpful since kernel updates typically have many CVEs to describe in the USN. You give it the CVE list that is in new-usn.sh and it will output example text that you can paste into new-usn.sh and edit. Eg:

    $ cd $UCT
    $ $UCT/scripts/pull-usn-desc.py --cve CVE-... --cve CVE-...
  • The title for the USN should be 'Linux kernel vulnerabilities'
  • The summary for the USN should be something like 'linux, linux-{ec2,fsl-imx51,mvl-dove,source-2.6.15,ti-omap} vulnerabilities'
  • In the minimum binaries section of the USN, use linux-image* (but no debug or dbgsym packages)

Mozilla

TODO

Chromium Browser

TODO

SecurityTeam/PublicationNotes (last edited 2023-04-13 16:58:52 by leosilvab)