Ubuntu Wiki

Dev Week -- Introduction to the Ubuntu Security Team - Kees Cook and Jamie Strandboge -- Fri, Sep 5

(03:04:27 PM) kees: I'll go ahead and get started.  As usual, please ask questions in the -chat room, and we'll answer them as we see them.  :)
(03:04:38 PM) kees: Hello!  I'm Kees Cook, and I'm the technical lead of the Ubuntu Security Team (and employed by Canonical).
(03:04:54 PM) kees: This is going to be an introduction to the Security Team, and things we're working on.
(03:05:07 PM) kees: I'm here with Jamie Strangeboge and William Grant.  We're going to trade off talking about various topics.
(03:05:29 PM) jdstrand: Strageboge?
(03:05:34 PM) kees: gah
(03:05:34 PM) jdstrand: Strangeboge?
(03:05:40 PM) jdstrand: Strandboge :P
(03:05:54 PM) kees: Strandboge.  apologies.  I swear I can type.  :)
(03:06:02 PM) ***jdstrand guesses he knows what kees thinks of him!
(03:06:10 PM) ***kees hangs his head in shame
(03:06:15 PM) kees: The Ubuntu Security Team is made up of the teams handling main, universe, and those working on pro-active hardening, as well as security auditing.  (See https://wiki.ubuntu.com/SecurityTeam/GettingInvolved)
(03:06:17 PM) jdstrand: heh, np
(03:07:24 PM) kees: First, I'm going to cover the "life cycle" of a security issue.  This is useful to understand for a developer, so it's obvious where things fit together.
(03:07:54 PM) kees: A security issue starts either with a bug reported to Launchpad, or as a "CVE" (http://cve.mitre.org/).
(03:08:28 PM) kees: For anyone unfamiliar with CVEs, it is maybe easiest to think of them as "global" bug reports.  :)
(03:08:35 PM) mcas_away is now known as mcas
(03:08:48 PM) kees: Once the bug is understood, we try to coordinate with upstreams or other distros to develop a patch.
(03:09:09 PM) kees: This is the first major bit of work -- actually _fixing_ the problem.
(03:09:29 PM) kees: As with SRUs, we try to produce a minimal change that fixes the problem.
(03:09:46 PM) kees: The patch is tested, and we then follow the "Security Update Procedures" and get it published.  (https://wiki.ubuntu.com/SecurityUpdateProcedures)
(03:10:07 PM) kees: This works much like a Stable Release Update (https://wiki.ubuntu.com/StableReleaseUpdates), and involves potentially even more careful testing.
(03:10:57 PM) kees: when doing these tests, the people involve will try to test out anything changed in the code, and make sure it both fixes the problems and doesn't break anything that used to work.
(03:12:18 PM) kees: when security updates are published for packages in main (and restricted), an Ubuntu Security Notice is published, outlining what was fixed.
(03:12:29 PM) kees: Those are seen here: http://www.ubuntu.com/usn/
(03:12:53 PM) kees: For anyone interested in getting these updates, there is a mailing list (ubuntu-security-announce) linked from the above page.
(03:13:42 PM) kees: The primary place where issues are tracked is in the Ubuntu CVE Tracker (https://launchpad.net/ubuntu-cve-tracker)
(03:14:10 PM) kees: It contains information about all the CVEs that impact Ubuntu, past and present.
(03:14:55 PM) kees: Since not everyone is interested in digging into a bzr repo just to see how things look, it is also published: http://people.ubuntu.com/~ubuntu-security/cve/main.html
(03:16:17 PM) kees: and for individual CVEs, those can be examined too: http://people.ubuntu.com/~ubuntu-security/cve/CVE-2008-2327
(03:16:18 PM) ubot5: kees: Multiple buffer underflows in the (1) LZWDecode and (2) LZWDecodeCompat functions in tif_lzw.c in the LZW decoder in LibTIFF 3.8.2 and earlier allow context-dependent attackers to execute arbitrary code via a crafted TIFF file.  NOTE: some of these details are obtained from third party information. (http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2327)
(03:16:27 PM) kees: (thanks ubot5)
(03:17:24 PM) kees: In addition to fixing security issues as they come up, we're also doing pro-active work to make security issues less of a problem when they happen.
(03:18:11 PM) kees: These mitigation techniques are wide-ranging including memory protections, mandatory access control (AppArmor and SELinux), firewalls (ufw), etc.
(03:18:39 PM) kees: the toolchain hardening options can be seen here: https://wiki.ubuntu.com/CompilerFlags
(03:19:01 PM) kees: many are new for Intrepid, but Edgy and later has had the stack protector.
(03:19:43 PM) kees: AppArmor and SELinux are available (AppArmor by default), and I'll let jdstrand talk about ufw shortly.
(03:19:47 PM) kees: QUESTION: how about security issues in universe and multiverse? it seems that security team is not issue announcements about it
(03:20:12 PM) kees: The Universe Security Team (motu-swat) handles updates for universe and multiverse
(03:20:32 PM) kees: (see http://people.ubuntu.com/~ubuntu-security/cve/universe.html)
(03:21:01 PM) kees: as of now, no one has stepped up to handle writing a "Universe USN" for updates that get published.
(03:21:34 PM) kees: I can let wgrant discuss this -- he is (hopefully) coming for the end of this class.
(03:22:07 PM) kees: Help with universe updates is greatly appreciated -- the above link shows which packages need work.
(03:22:38 PM) kees: I'll let jdstrand take over now....  :)
(03:22:45 PM) jdstrand: thanks kees!
(03:22:55 PM) jdstrand: Hi! My name is Jamie Strandboge, and I am a member of the Ubuntu Security Team, a Canonical employee, author of UFW, contributor to qa-regression-testing, and a whole bunch of other stuff noone probably cares about. :)
(03:23:21 PM) jdstrand: I'm giong to talk about qa-regression-testing and ufw
(03:23:33 PM) jdstrand: When performing a security update, it is of utmost importance to make sure that the update does not introduce any regressions and verify that the package works as intended after an update.
(03:23:55 PM) jdstrand: This is where the QA Regression Testing bzr branch (https://code.launchpad.net/~ubuntu-bugcontrol/qa-regression-testing/master) can help. qa-regression-testing was started by Martin Pitt (pitti), and continued by me, kees and others.
(03:24:16 PM) jdstrand: qa-regression-testing is used extensively by the Ubuntu Security team, as well as the Ubuntu QA Team, Ubuntu Server Team and others. They are also used in the SRU (Stable Release Update) process and when testing Apparmor profiles.
(03:24:47 PM) jdstrand: The bzr branch contains a lot of information to help with an update. I highly recommend reading README.testing, which talks about things to look out for in an update, best practices, checklists and more.
(03:25:03 PM) jdstrand: Also, the build_testing/ and notes_testing/ have notes and instructions on how to enable build testing, use testing frameworks for a particular application and any other notes pertinent to testing.
(03:25:21 PM) jdstrand: The scripts/ directory contains scripts for testing various programs. The main idea behind these scripts not build/compile testing, but rather application testing for default and non-default configurations of packages.
(03:25:41 PM) jdstrand: For example, the test-openldap.py script will test slapd for various configurations like ldap://, ldapi://, ldaps://, sasl, overlays, different backends and even kerberos integration.
(03:25:57 PM) jdstrand: *IMPORTANT* the scripts in the scripts/ directory are destructive, and should NOT be run on a production machine. We typically run these in a virtual machine, but often a chroot is sufficient.
(03:26:17 PM) jdstrand: Most of the scripts use python-unit. At the top of each script are instructions for how to use it, caveats, etc. There is also a skeleton.py script along with libraries (testlib*.py) that can be used when developing new scripts.
(03:26:39 PM) jdstrand: The scripts in qa-regression-testing typically are written when there is a new security update, and specifically tests the functionality that pertained to a given patch. As such, the scripts are in varying states of completeness, and any help in creating and extending these is most welcome. :)
(03:26:58 PM) jdstrand: By following the checklists, best practices, developing new scripts and using existing scripts for qa-regression-testing, we all can go a long way in helping to ensure as few regressions as possible.
(03:27:42 PM) jdstrand: I'm going to continue on with ufw now. if there are any questions, they can also be asked at the end of the session
(03:27:51 PM) jdstrand: ufw is Ubuntu's default firewall application, and as of Ubuntu 8.04 LTS (Hardy Heron), it is installed by default, but not enabled.
(03:28:04 PM) jdstrand: ufw stands for 'Uncomplicated Firewall', and strives to make configuration of an iptables firewall easier for users while not getting in the way of administrators with advanced needs.
(03:28:22 PM) jdstrand: Currently, it works very well as a host-based/bastion host firewall, particularly for desktop, laptop and single-homed servers.
(03:28:37 PM) jdstrand: Some of its features include:
(03:28:41 PM) jdstrand: * easy to disable and enable
(03:28:42 PM) jdstrand: * status and logging commands
(03:28:42 PM) jdstrand: * simple and extended rule syntax for allowing and denying traffic
(03:28:49 PM) jdstrand: * ipv4 and ipv6 support
(03:28:49 PM) jdstrand: * boot integration
(03:28:50 PM) jdstrand: * sysctl/proc integration
(03:28:57 PM) jdstrand: * reasonable defaults
(03:28:57 PM) jdstrand: * can add/delete/modify rules before enabling the firewall
(03:28:58 PM) jdstrand: * supports default DROP and default ACCEPT
(03:29:04 PM) jdstrand: * checks /etc/services for non-numeric ports
(03:29:21 PM) jdstrand: and as of Ubuntu 8.10 (Intrepid Ibex), ufw adds:
(03:29:33 PM) jdstrand: * connection rate limiting via the 'limit' command
(03:29:33 PM) jdstrand: * localization support
(03:29:34 PM) jdstrand: * port ranges (aka multiport) support
(03:29:41 PM) jdstrand: * dotted netmask support
(03:29:41 PM) jdstrand: * modularized code for better integration and downstream support (eg gui-ufw)
(03:29:45 PM) jdstrand: * application integration (aka package integration)
(03:30:12 PM) jdstrand: QUESTION: how about NAT in ufw?
(03:30:55 PM) jdstrand: I'm going to address that a little later on. the short answer is that the 'ufw' cli command doesn't do NAT, but that ufw framework allows you to do whatever iptables can do
(03:31:08 PM) jdstrand: Using ufw is pretty straightforward, and for the casual laptop or desktop user, it is simply a matter of running:
(03:31:11 PM) jdstrand: $ sudo ufw enable
(03:31:22 PM) jdstrand: This will drop incoming connections and allow all outgoing with connection tracking. It also makes sure that things like dhcp and avahi work, as well as load different connection tracking helper modules for ftp and irc. It also prevents logging of particularly noisy services (like CIFS)
(03:31:40 PM) jdstrand: You then can add new rules via the command line:
(03:31:41 PM) jdstrand: $ sudo ufw allow http
(03:31:42 PM) jdstrand: $ sudo ufw limit from 192.168.0.0/16 port 22 proto tcp
(03:31:53 PM) jdstrand: oops
(03:32:06 PM) jdstrand: $ sudo ufw limit from 192.168.0.0/16 to any port 22 proto tcp
(03:32:21 PM) jdstrand: and delete rules with:
(03:32:21 PM) jdstrand: $ sudo ufw delete allow http
(03:32:30 PM) jdstrand: $ sudo ufw delete limit from 192.168.0.0/16 to any port 22 proto tcp
(03:32:43 PM) jdstrand: You can also see the status of the ufw added rules in the running firewall
(03:32:46 PM) jdstrand: with:
(03:32:51 PM) jdstrand: $ sudo ufw status
(03:32:51 PM) jdstrand: Status: loaded
(03:32:51 PM) jdstrand: To                         Action  From
(03:32:51 PM) jdstrand: --                         ------  ----
(03:32:52 PM) jdstrand: 22/tcp                     ALLOW   192.168.2.0/24
(03:33:29 PM) jdstrand: QUESTION: why ufw is adding both TCP and UDP if not specified?
(03:33:53 PM) jdstrand: well, it doesn't know which you want unless you specify it
(03:34:10 PM) jdstrand: however, ufw has integration with /etc/services, so you can do something like:
(03:34:15 PM) jdstrand: $ sudo ufw allow http
(03:34:36 PM) jdstrand: because /etc/services only defines tcp for port 80, ufw will only open tcp port 80
(03:35:09 PM) jdstrand: QUESTION: is there any shortcut to delete rules, instead of  writing entire rule?
(03:35:12 PM) jdstrand: no
(03:35:21 PM) jdstrand: What is interesting about adding rules via the ufw command is that they are added t the running firewall as well as saved to configuration files.
(03:35:32 PM) jdstrand: As such, adding and deleting rules typically does not require reloading of the firewall (but where a reload is needed, ufw handles it for you automatically).
(03:35:41 PM) jdstrand: New in the Intrepid Ibex is application integration. This allows packages to add profiles to ufw, which users can then reference by name.
(03:35:52 PM) jdstrand: For example, the apache package in Ubuntu declares three profiles-- Apache, Apache Secure, and Apache Full, which correspond to ports 80/tcp, 443/tcp and 80,433/tcp respectively. A user could then do:
(03:36:01 PM) jdstrand: $ sudo ufw allow 'Apache Full'
(03:36:09 PM) jdstrand: to open tcp ports 80 and 443. This a particularly handy with more complicated protocols like CIFS. Eg:
(03:36:12 PM) jdstrand: $ sudo ufw allow Samba
(03:36:19 PM) jdstrand: will open udp port 137 and 138 as well as tcp ports 139 and 445.
(03:36:29 PM) jdstrand: You can get arbitrarily complicated and mix and match application rules with regular rules by using the extended syntax:
(03:36:32 PM) jdstrand: $ sudo ufw allow to 192.168.2.3 app Apache from 192.168.0.0/16 port 80,1024:65535,8080
(03:36:36 PM) jdstrand: $ sudo ufw status
(03:36:36 PM) jdstrand: ...
(03:36:37 PM) jdstrand: 192.168.2.3 Apache         ALLOW   192.168.0.0/16 80,1024:65535,8080
(03:36:45 PM) jdstrand: $ sudo ufw status verbose
(03:36:45 PM) jdstrand: ...
(03:36:47 PM) jdstrand: 192.168.2.3 80/tcp (Apache) ALLOW   192.168.0.0/16 80,1024:65535,8080/tcp
(03:36:59 PM) jdstrand: You can see a list of available profiles with the 'app list' command. Eg:
(03:37:03 PM) jdstrand: $ sudo ufw app list
(03:37:05 PM) jdstrand: Available applications: Apache Apache Full Apache Secure CUPS OpenSSH
(03:37:11 PM) jdstrand: Applications that currently have ufw integration (Intrepid only) are apache, bind, cups, dovecot, openssh, postfix, and samba (thanks nxvl and didrocks!).
(03:37:20 PM) jdstrand: Please note that installing a package will *not* add any rules or open any ports on your firewall.
(03:37:32 PM) jdstrand: The 'ufw' cli command provides a lot of functionality, and it very useful for a lot of people, but sometimes more functionality is needed. ufw as a whole allows administrators to take advantage of ufw's ease of use and adjust the firewall as much as desired by using various iptables chains.
(03:37:49 PM) jdstrand: The ufw cli command manipulates the ufw[6]-user* chains, but administrators can also modify ufw[6]-before* and ufw[6]-after* chains via /etc/ufw/*.rules files.
(03:37:58 PM) jdstrand: Eg, an incoming ipv4 packet will traverse through ufw-before-input -> ufw-user-input -> ufw-after-input. So an admin can add NAT and forwarding rules to these chains, but still do things like 'ufw allow 25/tcp'.
(03:38:13 PM) jdstrand: Don't want avahi to be allowed? Adjust /etc/ufw/before*.rules.
(03:38:21 PM) jdstrand: Need to enable port forwarding and NAT in your virtual machines? Adjust /etc/ufw/before*.rules and /etc/ufw/sysctl.conf.
(03:38:29 PM) jdstrand: Want to do egress filtering or add different commenction tracking helper modules? You can do it. Anything you can do with ip[6]tables, you can do within the ufw framework.
(03:38:38 PM) jdstrand: The implementation achieves this by:
(03:38:42 PM) jdstrand: - using iptables-save/iptables-retore syntax in config files
(03:38:43 PM) jdstrand: - using 3 sets of chains-- before, user and after. Rules managed with the ufw command are added to the 'user' chains, with before and after chains configurable by administrator
(03:38:47 PM) jdstrand: - when possible, modifying the chains in place, rather than reloading the full ruleset, which reduces connection dropping
(03:38:50 PM) jdstrand: - uses iptables comments for application rules
(03:39:06 PM) jdstrand: Basically, ufw not only provides an easier way to deploy and use a firewall, it provides application integration with Ubuntu applications and a ready to use framework for administrators requiring advanced functionality.
(03:39:32 PM) jdstrand: QUESTION: why you chose to have uppercase in package name?
(03:39:47 PM) jdstrand: the package name can be whatever is in the supplied package profile
(03:40:07 PM) jdstrand: what is in there is typically the marketing name of the software
(03:40:10 PM) jdstrand: eg OpenSSH
(03:40:42 PM) jdstrand: and that's pretty much it for ufw. wgrant?
(03:41:01 PM) kees: I think wgrant is missing -- it's very very early in the morning for him.
(03:41:18 PM) kees: I'll add some more details about working with ubuntu-cve-tracker
(03:41:25 PM) mazaalai: jdstrand: tnx for ufw, it really useful, and make my life a lot easier
(03:41:35 PM) jdstrand: mazaalai: glad you like it! :)
(03:41:52 PM) kees: Once you have a local branch of ubuntu-cve-tracker, the first thing to do is read, surprisingly, the README file.  :)
(03:42:16 PM) kees: from there, the structure of the CVE files in active/, retired/, and ignored/ will be more clear.
(03:42:41 PM) kees: Anyone interested in helping triage CVEs and their impact on various Ubuntu releases is encouraged to join our efforts.
(03:42:55 PM) ***mazaalai rising hand
(03:44:30 PM) jdstrand: I forgot to mention something else wrt ufw
(03:44:49 PM) jdstrand: there is quite a bit of documentation on it, which can be seen:
(03:44:54 PM) jdstrand: https://help.ubuntu.com/8.04/serverguide/C/firewall.html (hardy)
(03:44:54 PM) jdstrand: http://doc.ubuntu.com/ubuntu/serverguide/C/firewall.html (intrepid)
(03:44:55 PM) jdstrand: https://wiki.ubuntu.com/UbuntuFirewall
(03:45:00 PM) jdstrand: and of course 'man ufw'
(03:45:16 PM) kees: for people interested in helping with any aspect of Ubuntu Security (be it ubuntu-cve-tracker, ufw, patching, etc), the #ubuntu-hardening IRC channel is the best place to coordinate and ask questions.
(03:45:39 PM) kees: And the SecurityTeam wiki has information (but needs some work too)
(03:45:55 PM) kees: That's all we've got prepared for today.  Are there any other questions?
(03:47:27 PM) kees: alright then, thanks!  Next up at 20:00 UTC will be Kernel Discussion with Ben Collins.  :)
(03:47:40 PM) jdstrand: Is there mentoring available for the security team -  or what would you recommend we do if we wanted to start  contributing?
(03:47:45 PM) jdstrand: kees: ^
(03:47:52 PM) jdstrand: I'll field it
(03:48:25 PM) jdstrand: basically, people wanting to contribute to the Ubuntu Security team can do so in any of the ways kees mentioned
(03:48:48 PM) jdstrand: if people are wanting to patch a package, then the best thing to do is discuss it in #ubuntu-motu
(03:49:11 PM) jdstrand: that way others from MOTU-Swat can guide you through the process
(03:49:43 PM) jdstrand: when the patch is ready, attach a debdiff that follow SecurityUpdateProcedures to a bug
(03:50:03 PM) jdstrand: kees or I will then review it, provide feedback and publish it
(03:50:50 PM) jdstrand: members of motu-swat as well as kees and I are available for questions and help when needed
(03:54:48 PM) jdstrand: QUESTION: with the new hardening options, how does Ubuntu compare to other distributions or free OSs?
(03:55:18 PM) kees: jdstrand: heh, good question
(03:56:08 PM) kees: Intrepid will basically be on par with with Fedora and RHEL.  In the past, not many of the compiler hardening options were enabled (it's a tricky problem for how Debian packages are built, compared to how RPMs are built)
(03:56:34 PM) kees: A major difference to Fedora is our use of AppArmor by default instead of SELinux.
(03:56:48 PM) kees: So on MAC systems, we're more like SuSE (which uses AppArmor)
(03:56:59 PM) jdstrand: is most or all of grsecurity now included in Ubuntu?
(03:57:31 PM) jdstrand: (or its functional equivalent)
(03:57:41 PM) kees: grsecurity has a lot of misc kerne hardening features.  many aren't appropriate for general use, though many people ask about PaX.
(03:58:06 PM) kees: most of the elements of PaX (namely Address Space Layout Randomization) are in the mainline linux kernel now, so everyone gets it.
(03:58:20 PM) kees: Fedora published this great chart: http://www.awe.com/mark/blog/200801070918.html
(03:58:46 PM) kees: discounting the SELinux bits, Intrepid can make the same claims as Fedora 8 in that chart.
(03:59:15 PM) kees: well, except NX emulation, which we don't think is worth the performance hit
(03:59:40 PM) jdstrand: to clarify, we do have apparmor, and selinux is now available as a viable option in Ubuntu
(03:59:58 PM) kees: okay, thanks again everyone!  we gotta clear out for BenC.  :)