Ubuntu Wiki

Open Week -- Proactive Security Demonstration -- Kees Cook -- Wed, May 5

EDT -5

(03:01:16 PM) akgraner: kees, if you are ready the floor is yours
(03:01:25 PM) kees: hello!
(03:01:48 PM) kees: this is going to be fun; I'll be using an EC2 ubuntu system to demo security features in Ubuntu.
(03:02:10 PM) kees: my name is Kees Cook, and I'm on the Ubuntu Security team.
(03:02:35 PM) kees: I'm going to give some examples of what security features ubuntu has, based on our list here:
(03:02:39 PM) kees: https://wiki.ubuntu.com/Security/Features
(03:03:15 PM) kees: before I get started, just a quick show of hands in the -chat channel, who is new to that page, and who has read it before?
(03:03:51 PM) ***kees checks for lurkers... :)
(03:04:46 PM) kees: okay, cool.  looks like we need to make more people aware of that page.  :)
(03:05:08 PM) kees: anyway, to get started with this, I'd like everyone to ssh to the EC2 instance I have running.
(03:05:15 PM) kees: ssh -C guest@ec2-184-72-13-215.us-west-1.compute.amazonaws.com
(03:05:26 PM) kees: the password is "guest", and it will drop you into a read-only "screen" session
(03:05:37 PM) kees: where you can follow along and watch me type commands, see output, etc.
(03:06:11 PM) kees: I'll pause here a minute to let people get online.  You should see my  Welcome to the Ubuntu Open Week "Proactive Security Demonstration"  banner in an 80x25 terminal once you're in.
(03:06:25 PM) kees: QUESTION: RSA key fingerprint is 4b:0d:08:3f:cd:7a:3f:ce:04:00:71:a8:c7:e6:b6:31?
(03:06:29 PM) kees: yup
(03:07:10 PM) kees: okay.  I'm going to be wandering around in the Features list, since some things are easier to demo that others.
(03:07:18 PM) kees: however, I wanted to call attention to our regression test suite:
(03:07:26 PM) kees: https://code.launchpad.net/~ubuntu-bugcontrol/qa-regression-testing/master
(03:07:38 PM) kees: it's large, so I've extracted the tests I'm going to use:
(03:07:41 PM) kees: http://people.canonical.com/~kees/demo/
(03:07:58 PM) kees: you can follow along on your own system if you want, or you can watch the EC2 session.
(03:08:17 PM) kees: basically, these tests attempt to validate as many of the Ubuntu security features as possible.
(03:09:11 PM) kees: (oh, also, please feel free to ask questions as we go, and I'll either answer right away, or say that I'm delaying it until later)
(03:11:06 PM) kees: okay.
(03:11:11 PM) kees: so, here goes in the EC2 session
(03:11:28 PM) kees: in the "qrt" directory are the tests I linked to above
(03:11:32 PM) ClassBot: nealmcb asked: can we somehow replay this demo for friends later?
(03:11:51 PM) kees: this test builds and tests a whole stack of kernel security tests. the next for glibc, and finally gcc (the compiler)
(03:12:42 PM) kees: now, each test can be examined, but I wanted to look at just a few.
(03:12:56 PM) kees: so, here's a quick one:  https://wiki.ubuntu.com/Security/Features#No%20Open%20Ports
(03:13:03 PM) kees: "what is listening by default?"
(03:13:13 PM) kees: netstat -lpwut
(03:13:16 PM) kees: will show that.
(03:13:40 PM) kees: I have apache and ssh listening because I installed them, and DHCP is there too.  so, yeah, it's all good.
(03:14:02 PM) kees: moving on: https://wiki.ubuntu.com/Security/Features#Hardening
(03:14:23 PM) kees: this is a large catch-all that the qrt stuff attempts to test.
(03:14:30 PM) kees: a direct example: https://wiki.ubuntu.com/Security/Features#Non-Exec%20Memory
(03:14:50 PM) kees: this is to see that data cannot be executed.  it stops a huge class of memory corruption vulnerabilities.
(03:15:08 PM) kees: in the "nx" subdirectory is a tool that specifically tries to execute code from data areas.
(03:15:23 PM) kees: we'll use the "stack" test, since it's the most well understood in general
(03:15:58 PM) kees: so, here we see it attempting to run code from the stack, but the kernel stops it.  You can see the report from "dmesg" about the failure.
(03:16:27 PM) kees: without "nx", it would just return.  you can see that with "mmap-exec", which is a way to set up a memory region to execute data...
(03:16:42 PM) kees: in this case, the test tool expects to return.
(03:17:34 PM) kees: so, even some attack manages to break a piece of software and tries to inject their code, they're going to have a hard time of it since there are very few places where the kernel won't just kill the program out-right.
(03:17:43 PM) kees: moving on: https://wiki.ubuntu.com/Security/Features#ASLR
(03:18:02 PM) kees: this randomizes process memory so an attack can't guess where to even begin their memory corruption attack.
(03:18:30 PM) kees: if I look at two instances of the "cat" process's stack location, I can see they're in different locations on two different runs.
(03:18:41 PM) kees: bfec5000 vs bf909000
(03:19:00 PM) kees: but, we also want: https://wiki.ubuntu.com/Security/Features#/proc/$pid/maps%20protection
(03:19:24 PM) kees: I shouldn't be able to see another user's memory maps (in this case, root's, for process 1 -- init)
(03:19:42 PM) kees: next: https://wiki.ubuntu.com/Security/Features#Stack%20Protector
(03:20:06 PM) kees: the Ubuntu toolchain enables the "stack protector" to detect overflows.
(03:20:26 PM) kees: without it, an overflow is uncontained, as we see with the stack-protector-off program
(03:20:39 PM) kees: if we look in gdb at this...
(03:21:00 PM) kees: we see it's trying to execute at 0x41414141.
(03:21:22 PM) kees: and that's "AAAA" (we can see from "man ascii")
(03:21:47 PM) kees: we don't want an attacker to be able to jump to arbitrary locations, so the stack protector (gcc's -fstack-protector) is always used.
(03:22:26 PM) kees: in the case of the -on binary, it shuts down instead of allowing the process to continue
(03:23:31 PM) kees: you can see that glibc is killing it off an screaming loudly "*** stack smashing detected ***"
(03:23:53 PM) kees: next example: https://wiki.ubuntu.com/Security/Features#Fortify%20Source
(03:23:54 PM) ClassBot: jdstrand asked: will you be making your screen session available online (eg via script of similar?)
(03:24:18 PM) kees: jdstrand: yup, I've got "script" running for that purpose.  :)
(03:24:32 PM) kees: so, fortify source is similar to the stack protector, but it covers a LOT of other things.
(03:24:48 PM) kees: I'm going to look only at "format string protection", since it's kind of wild.
(03:25:26 PM) kees: if you see during the compile, gcc yelled about an unsafe use of a format string.  i.e.  printf(variable); instead of printf("%s",variable");
(03:25:31 PM) kees: so we can run it...
(03:25:50 PM) kees: we see that the format string is being accidentally expanded.
(03:26:06 PM) kees: and in the case of "%n", this can allow an attack to write to memory locations.
(03:26:25 PM) kees: in this case, it's just exploding, but it could be manipulated.
(03:26:46 PM) kees: so, with the "fixed" version, it's still got the bug, but %n gets caught
(03:27:21 PM) kees: the fortify source protections are really cool, I think.  :)
(03:27:41 PM) kees: next up: https://wiki.ubuntu.com/Security/Features#0-address%20protection
(03:28:15 PM) kees: this got a lot of attention recently due to the popularization of "kernel NULL pointer attacks".  Ubuntu has been protected since 8.04 from this stuff.
(03:28:28 PM) kees: we can see the bottom limit of what memory can be used (64k here)
(03:28:50 PM) kees: and this test validates that it can't map memory below that 64k.
(03:29:25 PM) kees: if we try, we see that it rejects it.  no way for an attack to map low memory and then exploit a kernel bug to gain root there.  :)
(03:29:40 PM) kees: relatively new feature: https://wiki.ubuntu.com/Security/Features#Block%20module%20loading
(03:29:53 PM) kees: I called this out in particular because not a lot of people know about it.
(03:30:32 PM) kees: one way that attackers really make someone's life miserable is by injecting evil kernel modules once they're root.  (though really, if they're root, boy does that person have trouble)
(03:31:29 PM) kees: this /proc/sys setting controls if modules can be loaded at all.  I'm not actually going to change this setting since I'm not sure if EC2 to hate me afterwards.  :)
(03:31:46 PM) kees: if you set it to "1" it can't be set back to 0, so it's a one-way toggle for after you've booted a system.
(03:32:10 PM) kees: now, moving on to another slightly longer example.
(03:32:13 PM) kees: https://wiki.ubuntu.com/Security/Features#AppArmor
(03:32:25 PM) kees: this is the MAC system that can really tightly isolate processes.
(03:32:36 PM) kees: in this case, I'll confine an apache host.
(03:32:47 PM) kees: so, the demo server is running: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/
(03:33:08 PM) kees: and i've written an insane PHP script: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.phps
(03:33:22 PM) kees: it takes arguments, e.g. http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=info
(03:33:53 PM) kees: and we can see files with it: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=/etc/motd
(03:34:13 PM) kees: but since it's unsafe, we can run other stuff too: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=/dev/null;id
(03:34:37 PM) kees: so, to protect ourselves from this kind of evilness, we can enable the apache profile I wrote...
(03:35:14 PM) kees: so, here's a running apache instance with the profile active.
(03:35:38 PM) kees: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/ still works
(03:35:46 PM) kees: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=info  still works
(03:35:50 PM) kees: but this does not:
(03:35:55 PM) kees: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=/dev/null;id
(03:36:11 PM) kees: and we see the rejection from apparmor in the dmesg output.
(03:36:45 PM) kees: we could add back the shell (dash) and "cat", to see files again, but "id" would be blocked...
(03:37:24 PM) kees: let's see if I can do this live...
(03:38:30 PM) kees: http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=/etc/motd  works again
(03:38:46 PM) kees: and  http://ec2-184-72-13-215.us-west-1.compute.amazonaws.com/unsafe.php?q=/dev/null;id  doesn't.
(03:39:43 PM) kees: so, this has been a bit of a fire-hose of examples and script and such, but I wanted to leave some time here on the back-half for questions, or any areas I could drill down into, etc.
(03:40:28 PM) kees: anyone got any questions about this, the general security feature list, security in general, etc?
(03:41:27 PM) ClassBot: JR0cket asked: security newbie - all the security measures shown are standard in Ubuntu desktop and server?
(03:41:51 PM) kees: yes, everything here is standard to ubuntu and all the derivatives.
(03:42:17 PM) ClassBot: Resno asked: So, for general use what are things we can do to better secure ourselves from attacks?
(03:42:33 PM) kees: in case, the compiler hardening (stack protector, fortify, etc) are all built into the compiler so even locally compiled stuff gains the protections.
(03:42:52 PM) kees: in general, things are pretty safe.
(03:43:03 PM) kees: I tend not to trust my browser
(03:43:10 PM) kees: so I enable the AppArmor profile for firefox.
(03:43:27 PM) kees: you also want to make sure you've actually got NX working on your system.
(03:43:40 PM) ClassBot: sebsebseb asked: I don't really understand this session, but I know PHP scripts can be rather insecure when not coded properly,  so basically Apparmor can help secure such scripts on an Apache web server?
(03:44:15 PM) kees: that's one of many ways to use AppArmor, yes.  You'll want the libapache2-mod-apparmor package installed.
(03:44:30 PM) kees: and you'll want to examine /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2
(03:44:32 PM) ClassBot: CodyR asked: Why use AppArmor over Selinux?
(03:44:57 PM) kees: it includes /etc/apparmor.d/apache2.d/ where you can write custom profiles, etc.
(03:45:07 PM) kees: both SELinux and AppArmor work on Ubuntu
(03:45:16 PM) kees: we just use AppArmor by default because it's so much easier to use.
(03:45:33 PM) ClassBot: JR0cket asked: AppAmor looks great, but where can I see lots more (Ubuntu) examples?
(03:46:12 PM) kees: for apparmor in general, see /etc/apparmor.d/ all the default profiles are in there.  for more the "apparmor-profiles" package has some examples
(03:46:30 PM) kees: there's information about writing profiles too
(03:46:41 PM) kees: see https://help.ubuntu.com/community/AppArmor
(03:46:50 PM) ClassBot: yltsrc asked: howto get/enable firefox profile?
(03:47:34 PM) kees: oh! yes, it comes disabled by default, but:  sudo rm /etc/apparmor.d/disable/*firefox*; sudo service apparmor reload   will do it
(03:47:44 PM) kees: be sure to read the profile; it blocks a lot of stuff.  :)
(03:48:07 PM) ClassBot: yltsrc asked: apparmor profiles for chromium|epiphany|etc.?
(03:48:42 PM) kees: there aren't yet, but at UDS, we plan on working on chromium a bit.
(03:48:54 PM) ClassBot: CodyR asked: Is AppArmor better or worse than other similar mechanisms such as SELinux / GRSecurity kernel patch? Does it offer anything else / additional?
(03:49:10 PM) kees: as jdstrand mentioned in -chat, for the firefox profile, this is probably easier:  sudo aa-enforce /etc/apparmor.d/usr.bin.firefox
(03:49:33 PM) kees: AA is just different from SELinux and grsecurity's RSBAC
(03:49:58 PM) kees: AA and RSBAC try to confine process, and SELinux tries to confine data, is how I think of it.
(03:50:00 PM) ClassBot: sebsebseb asked: What firewall to use  in Ubuntu is the kind of question that will come up in #ubuntu every now and again.  Then it's like you can learn how to configure iptables or install a graphical firewall such as gufw and enable it.  For a home user is there really a need to do anything when it comes to firewall and Ubuntu?  Is it enough to rely on whatever is set up by default in Ubuntu for security, and a built in router firewall.
(03:50:14 PM) kees: it's inaccurate on all counts, but the differences are subtle.
(03:50:48 PM) ClassBot: virtuald asked: ?how do firefox apparmor profile work with extensions and plugins?
(03:51:13 PM) kees: I think firewalls tend to get in the way of things, and generally don't really help security when you're running services.  firewalls don't block a bug in a web server, for example.
(03:51:29 PM) kees: but, if you want to keep safe anyway, ufw (or gufw) is _great_.
(03:51:45 PM) kees: I think learning to use ufw is way easier than iptables directly.
(03:52:23 PM) kees: "how do firefox apparmor profile work with extensions and plugins?"
(03:52:49 PM) kees: if the plugin doesn't try to access any files outside of ~/.mozilla or run any extra programs, it usually works just fine.
(03:53:22 PM) kees: that said, you'll want to be running "aa-notify" or watching your dmesg output for hints when things get blocked that you didn't want to be blocked.
(03:54:27 PM) kees: sounds like we're out of questions for now.  if you think of any more or generally want hang out, please join us in #ubuntu-hardened on IRC or feel free to email me at kees@ubuntu.com.
(03:54:38 PM) kees: thanks everyone for joining in and participating!
(03:54:50 PM) akgraner: Thanks kees!
(03:54:55 PM) ClassBot: huntz0r asked: Nice session, but a lot of it went way over my head! :-)   is there any place you would recommend to help swot up on some of the things covered?
(03:55:09 PM) kees: https://wiki.ubuntu.com/Security/Features has a lot of pointers
(03:55:12 PM) kees: I would also read:
(03:55:18 PM) kees: https://wiki.ubuntu.com/CompilerFlags
(03:55:33 PM) ClassBot: nealmcb asked: do you have any way of knowing which features matter the most in the wild?
(03:55:45 PM) kees: yes.  NX is without a doubt, #1.
(03:56:17 PM) kees: there are tons of logic mistakes in webservers and scripts, but NX will block a lot of further escalation
(03:56:20 PM) ClassBot: yltsrc asked: is it possible to setup aa-notify to send emails?
(03:56:36 PM) kees: I'd put a MAC system, like AppArmor at #2 for servers.
(03:56:47 PM) kees: but then fortify+stack protector, etc.
(03:57:00 PM) ClassBot: JR0cket asked: are all new apparmor profiles packaged in the apparmor-profiles package, or will they be seperated out? something for UDS?
(03:57:27 PM) kees: we separate profiles into the package that ships the binary.  apparmor-profiles is really just for non-production testing
(03:57:53 PM) kees: 19:56 <+ClassBot> yltsrc asked: is it possible to setup aa-notify to send emails?
(03:58:06 PM) kees: not presently, but I bet it wouldn't be too hard to add.  :)
(03:58:26 PM) kees: okay, I'm out of time!  thanks again everyone!
(03:58:53 PM) akgraner: Thanks again kees and everyone who joined the session