Ubuntu Wiki

   1 [19:00] <Daviey> Next we have kees_lernid for an exciting, and cool subject!
   2 [19:01] <kees> well, just me, not kees_lernid, that's just me following along
   3 [19:01] <kees> Hello!  I'm Kees Cook, an Ubuntu Developer, Security Team member, and Canonical employee.
   4 [19:01] <kees> I'm trying to figure out how to get my slides into lernid...
   5 [19:02] <kees> This presentation is about how to run the development version of Ubuntu (currently Lucid) and how to get yourself out of jams.
   6 [19:02] <kees> The slides I'll be using to keep me on topic come from 2007, and I'll update a few links as I go.
   7 [19:02] <kees> Nearly everything still applies, but just mentally swap in "Lucid" when you see "Gutsy".  :)
   8 [19:03] <kees> They are here: http://outflux.net/ul07/bleeding-edge.pdf (or .odp if you want the raw slides)
   9 [19:03]  * kees attempts to trigger lernid somehow...
  10 [19:03] <kees> [SLIDES: http://outflux.net/ul07/bleeding-edge.pdf]
  11 [19:03] <kees> dang
  12 [19:03] <kees> http://outflux.net/ul07/bleeding-edge.pdf
  13 [19:04] <kees> one moment, trying to get the slides into iCal...
  14 [19:04] <kees> [SLIDE 1]
  15 [19:04] <kees> hrm
  16 [19:06] <kees> okay... so, this should work, or reload lernid:  wget -O $HOME/.cache/lernid/slides.pdf http://outflux.net/ul07/bleeding-edge.pdf
  17 [19:06] <kees> anyway, no more delays.  :)
  18 [19:06] <kees> [SLIDE 1]
  19 [19:06] <kees> The bulk of this content was written by cjwatson; I just cleaned them up.  Blame me for any mistakes, though.  :)
  20 [19:06] <kees> [SLIDE 2]
  21 [19:07] <kees> so, people like running the devel release, but there are a lot of reasons not to
  22 [19:07] <kees> in particular, if you need a new package, and there is no PPA for it, you can just build it on a stable release
  23 [19:07] <kees> the only thing you need to find generally is the ".dsc" file, and "dget" will do the work of fetching it all
  24 [19:08] <kees> in this slide, I show the commands to do a package build; it's a few steps, but it gets it done, and gives you a working package usually.
  25 [19:08] <kees> [SLIDE 3]
  26 [19:08] <kees> now, there are reasons TO run the devel release; and I love having people helping with it.
  27 [19:08] <kees> the biggest reason is helping with development and testing.
  28 [19:09] <kees> it's very hard to do testing without actually running the software you're working on.  :)
  29 [19:09] <kees> generally, though, if you're interested in this topic, you should be running the devel release.  :)
  30 [19:10] <kees> oh! before I forget, please feel free to ask questions with [QUESTION] in the #ubuntu-classroom-chat channel as I go.
  31 [19:10] <kees> [SLIDE 4]
  32 [19:10] <kees> so, we're working on Lucid now, so you'll want https://wiki.ubuntu.com/LucidReleaseSchedule
  33 [19:11] <kees> prior to import freeze, we're pulling in new software versions from Debian daily.
  34 [19:11] <kees> and people are, of course, working on all kinds of fixes and changes.
  35 [19:11] <kees> that said, we stop for regular milestones and release bootable and installable "alpha" liveCDs.
  36 [19:12] <kees> after feature freeze, we try to focus on bug fixing.  with Lucid being an LTS, we're actually trying to focus more on bug fixing than features too.
  37 [19:12] <kees> it's really important to get testing in before the end of this schedule, usually before beta because it's very hard to fix some bugs without enough time to test them.
  38 [19:12] <kees> [SLIDE 5]
  39 === nathan is now known as Guest75246
  40 [19:14] <kees> given how much the release changes from day to day, it's important to stay updated.  I try to update twice a week.  Others do it multiple times a day.  I would recommend at least once a week.  I find Wed the best time so that if things have gone weird, there are a weekdays left before the weekend to try to get help with fixing stuff.
  41 [19:14] <kees> following that rule, I don't update on fridays.  ;)
  42 [19:14] <kees> please help out with the testing and liveCD testing.  the included links in the slides are good starting points.
  43 [19:14] <kees> [SLIDE 6]
  44 [19:15] <kees> so, the liveCDs are important.  they're the best indicator of what the ubuntu install experience is going to be.
  45 [19:15] <kees> you can also build bootable USB disks from liveCD isos
  46 [19:15] <kees> or, with a VM, you can boot isos for testing.
  47 [19:15] <kees> my favorite for this is testdrive: http://blog.dustinkirkland.com/2009/11/introducing-testdrive.html
  48 [19:16] <kees> it lets you trivially try out an ubuntu image if you don't want to install it or be running the devel release
  49 [19:17] <kees> the USB live disk details are probably best described here: http://en.wikipedia.org/wiki/Ubuntu_Live_USB_creator
  50 [19:17] <kees> [SLIDE 7]
  51 [19:17] <kees> another option is dual-booting, a separate system, etc.
  52 [19:18] <kees> [SLIDE 8]
  53 [19:18] <kees> I'm personally a fan of using VMs.  libvirt and "virt-manager" is my friend.  All my VMs are there, though it does start taking up a fair bit of disk space.
  54 [19:18] <kees> $ du -sh /vm-images/
  55 [19:18] <kees> 84G      /vm-images/
  56 [19:18] <kees> that said, I have amd64 and i386 of every stable release.  :)
  57 [19:19] <kees> [SLIDE 9]
  58 [19:19] <kees> now, with all the focus on install testing, upgrade testing is sometimes overlooked
  59 [19:19] <kees> upgrading a VM from the prior stable release to the devel release needs to be tested often, since there are library changes, package relationship changes, etc, that might need to be manually tweaked by update-manager.
  60 [19:20] <kees> filing bugs about this area is very important.
  61 [19:20] <kees> [SLIDE 10]
  62 [19:20] <kees> the "right" way to do upgrades is with update-manager.
  63 [19:20] <kees> it has additional logic that apt-get and aptitude do not contain.
  64 [19:21] <cjohnston> < strycore66> QUESTION : Is it good practice to keep a pristine VM for bug testing (updated but with no additonal packages or data) ?
  65 [19:21] <kees> if you get into a jam, you may need to using apt-get manually.  After a dist-upgrade, I also usually try an "apt-get install ubuntu-desktop" in case the manual upgrade missed anything
  66 [19:22] <kees> strycore66: so, I try to do both.  A pristine VM is good for being able to compare between version of Ubuntu, for example, but a "messy" VM is good for finding corner-cases, etc.
  67 [19:22] <kees> doing testing on your real system is best because you've customized it, and that's usually where bugs come from.
  68 [19:22] <kees> that said, it's good to start from the same pristine VM when you're trying to reproduce problems, etc.
  69 [19:22] <cjohnston> < alvarezp> QUESTION : What is that additional logic that update-manager provides? What do we miss if we use plain APT tools?
  70 [19:22] <kees> the "virt-clone" command is handy for this.
  71 [19:24] <kees> alvarezp: there are sometimes transititions or file clean ups that are hard to handle strictly in apt.  things I'm thinking of off the top of my head are: old kernel version cleanups, the open-office version bump a while back, stuff like that.
  72 [19:24] <kees> I don't have very good examples, but if you want more, find "mvo" online in #ubuntu-devel later and ask him, he tends to be the person handling those things for update-manager.
  73 [19:24] <kees> [SLIDE 11]
  74 [19:25] <kees> there's a lot that can break...
  75 [19:25] <kees> hopefully the rest of this presentation can help you with those issues
  76 [19:25] <kees> [SLIDE 12]
  77 [19:25] <cjohnston> < n3rd> QUESTION: What if i want to take a tgz of "/" and restore it ?
  78 [19:25] <kees> n3rd: let me answer that in a moment
  79 [19:26] <kees> so, if mirrors blow up, there can be checksum failures.  don't ignore these issues; they're real problems.
  80 [19:26] <kees> wait for the next update push (about an hour usually) and try again
  81 [19:26] <kees> or switch to the non-mirror briefly.
  82 [19:27] <kees> n3rd: you can take tarball snapshots of / if you want.  the main areas of package management are in /var/lib/dpkg and /var/lib/apt
  83 [19:27] <kees> I'll mention those in a moment
  84 [19:27] <kees> [SLIDE 13]
  85 [19:27] <kees> sometimes there are mistakes made when packages take over files from older packages.  all of these glitches need to be reported so they can be fixed.
  86 [19:27] <kees> [SLIDE 14]
  87 [19:28] <kees> you can force installations anyway, but if maintainer scripts misbehave, you'll likely need to debug and edit them by hand
  88 [19:28] <kees> the important bit is /var/lig/dpkg/info/ which contains all the maintainer scripts.
  89 [19:29] <kees> my favorite reference for how maintainer scripts are calls is here: http://wiki.debian.org/MaintainerScripts
  90 [19:29] <kees> the "Upgrading" diagram is extremely helpful in understanding when/how the scripts are run
  91 [19:29] <kees> [SLIDE 15]
  92 [19:30] <kees> after unpack and install, the postinst runs, and if it blows up, it'll leave the package unconfigured.  you may need to edit it and re-configure.
  93 [19:30] <kees> (again, always file a bug if you run into this kind of thing)
  94 [19:30] <kees> [SLIDE 16]
  95 [19:30] <kees> sometimes debugging maintainer scripts is a pain.
  96 [19:31] <kees> adding a "set -x" to maintainer scripts can help you see what it's trying to do.  some fail very silently, and this is probably the simplest way to get insight into how they're working
  97 [19:31] <kees> [SLIDE 17]
  98 [19:32] <kees> when something goes wrong during interactive questions (debconf), you can enable debugging to see what debconf is up to.
  99 [19:32] <kees> soemtimes a debconf item is misnamed, or wasn't created yet.
 100 [19:32] <kees> these things are also in /var/lib/dpkg/info, with the .config and .template extensions
 101 [19:32] <kees> [SLIDE 18]
 102 [19:33] <kees> in the worst-case situation, maybe dpkg itself is doing something wrong.  in that case, break out the big guns and run dpkg under strace to see all the syscalls.
 103 [19:33] <kees> I like adding "-s 1024" to strace so I can see some more details on "exec" and "open" calls.
 104 [19:34] <kees> dpkg has debugging options too, but they're really only for dpkg itself.
 105 [19:34] <kees> [SLIDE 19]
 106 [19:34] <kees> if you're prompted to remove stuff, just be careful.  :)
 107 [19:34] <kees> if you ever see "essential packages will be removed" just say no.  :)
 108 [19:35] <kees> you will almost always wreck your system and need extensive rescue CD help if you drop essential packages.
 109 [19:36] <kees> I really like "apt-cache policy [packagename]" to tell me about where a package came from, what versions are available, etc
 110 [19:36] <kees> really handy for debugging issues with software from other PPAs, etc
 111 [19:36] <kees> it only looks at your apt sources, which is handy.
 112 [19:37] <kees> another tool I like for undoing insanity is downgrade-all: http://bazaar.launchpad.net/~ubuntu-security/ubuntu-security-tools/trunk/annotate/head%3A/utilities/downgrade-all
 113 [19:37] <kees> this will remove all the third party versions of everything.
 114 [19:37] <kees> use it carefully, but I love it for restoring a system back to a known-good state.  read the downgrade/removal list _carefully_ if you use it.
 115 [19:38] <kees> [SLIDE 20]
 116 [19:38] <kees> if you need to force a removal, you can do that, but it's best to tell apt-get to attempt to get things back to normal afterwards (apt-get -f install)
 117 [19:39] <kees> sometimes 3rd party software can get in the way of an upgrade.  or in the way of a normally operating system.
 118 [19:39] <kees> recently I saw someone trying to run the Debian experimental ifupdown package.  if lacked the needed Ubuntu modifications, so it rendered their networking unusable.
 119 [19:39] <kees> downgrade-all fixed that.  :)
 120 [19:39] <kees> [SLIDE 21]
 121 [19:40] <mhall119|work> < Omar87> [QUESTION] Why do I always get partial upgrades only?
 122 [19:40] <kees> so, if you lose power in the middle of an install, or the system hangs, you'll need to try to get dpkg and apt to finish the job once you're booted back up.
 123 [19:40] <kees> dpkg --configure -a  is the first step to handle any unconfigured packages
 124 [19:41] <kees> and then apt-get -f install  to finish any installations
 125 [19:41] <kees> Omar87: partial upgrades?  if that's from update-manager, it's been conservative about doing package upgrades.  (it was mostly made to handle stable release updates and full-distro-upgrades).  For day-to-day use of the devel release, I recommend just "apt-get dist-upgrade"
 126 [19:42] <kees> if a package is really hosed, you can manually install it with dpkg
 127 [19:42] <kees> if the package _database_ gets hosed, you need to examine /var/lib/dpkg/ and /var/backups
 128 [19:42] <kees> the status and status-old files are imporant.
 129 [19:42] <kees> *important
 130 [19:43] <kees> if "status" is corrupted, try using prior versions in its place (status-old, then stuff in /var/backups)
 131 [19:43] <kees> I've had to manually repair my status file before when XFS tried to eat my harddrive.  it's a bit crazy, but once you have something mostly sane in "status" again, you can just reinstall any weird or missing packages
 132 [19:43] <kees> [SLIDE 22]
 133 [19:44] <kees> if your status file is out of date with what actually got installed, you can look at /var/log/dpkg.log and find the package that were installed since your last good status file.
 134 [19:44] <kees> then you can just do an  apt-get install --reinstall $package   for the stuff that got missed.
 135 [19:45] <kees> when I was trying to restore sanity to a particularly busted machine, I repairs the status file, used downgrade-all, and then did a --reinstall on all the package on the system.
 136 [19:45] <kees> took a while, but it was mostly ok after that.
 137 [19:46] <kees> dpkg-reconfigure is handy too if a question gets missed or corrupted.
 138 [19:46] <kees> [SLIDE 23]
 139 [19:46] <kees> in the case of hardware-specific breakage, it gets harder to have people helping you because they likely don't have your hardware
 140 [19:47] <kees> video and network are the trouble-areas here.
 141 [19:47] <kees> I tend to boot older kernels or try earlier Xorg versions.  sometimes you'll need to help find the problem by "bisecting" a source branch and building several versions of a package.
 142 [19:47] <kees> having a good bug report with all the hardware details helps a great deal.
 143 [19:48] <kees> [SLIDE 24]
 144 [19:48] <kees> when the ABI of a kernel changes, you'll have the older one still installed.  handy to boot from that when a newer kernel breaks something
 145 [19:48] <cjohnston> < rmunn> QUESTION: Any advice on how best to report hardware-related bugs? Specifically, which log file(s) would likely be useful to put in the bug report?
 146 [19:49] <kees> "apport" and the "ubuntu-bug" tools are very handy for reporting all kinds issues, but especially good for the kernel.
 147 [19:49] <kees> rmunn_: answering that now.  :)
 148 [19:49] <kees> running "ubuntu-bug linux" will gather dmesg output, lspci output, etc.
 149 [19:50] <kees> the "ubuntu-bug" tool tries to include everything that is mentioned on the various wiki pages for Debuging____
 150 [19:50] <kees> dmesg and lspci tends to be the most important pieces
 151 [19:50] <kees> we also have the kerneloops handler too.
 152 [19:50] <kees> [SLIDE 25]
 153 [19:51] <kees> when X breaks, it's time to start including all sorts of logs.  again. ubuntu-bug is your friend.  :)  "ubuntu-bug xorg-server" will tend to collect many of the needed pieces
 154 [19:51] <kees> falling back to VESA can help get you back up and running in the meantime.  /var/log/gdm may have clues
 155 [19:52] <cjohnston> < n3rd> QUESTION:xorg.conf why no such file in Karmic ? And is there any open drivers for ATI Technologies Inc Radeon HD 3200 Graphics?
 156 [19:52] <kees> [SLIDE 26]
 157 [19:52] <kees> this slide is out of date.  NetworkManager is in /etc/init now, so "service network-manager stop" is the way to disable it
 158 [19:53] <kees> configuration in /etc/networking/interfaces is used by the "networking", "networkinger-interface", and "network-manager" services, so it's the best place to start.
 159 [19:53] <kees> also, network names are remembered in /etc/udev/rules.d/70-persistent-net.rules
 160 [19:54] <kees> n3rd: xorg.conf isn't really needed any more because X can autodetect almost everything now.
 161 [19:54] <kees> having an xorg.conf almost always causes X to misdetect stuff, so instead, it's just been removed.
 162 [19:54] <kees> [SLIDE 27]
 163 [19:54] <cjohnston> < rmunn> QUESTION: I've had a hard time finding good documentation on network-manager and how it relates to (say) /etc/network/interfaces and the like. Any recommended reading?
 164 [19:55] <kees> in a really tight pinch, you can get into your system by changing the grub boot line to init=/bin/sh or by adding "break=mount" to land in the initramfs to debug issues with booting
 165 [19:55] <kees> I got very familiar with this while debugging RAID and LVM support :)
 166 [19:56] <kees> rmunn: the problem with network-manager is that its integration with Debian-like systems has always been troublesome.  each release it behaves slightly differently, which makes good documentation hard to find.  :(
 167 [19:57] <kees> I recommend asking "asac" in #ubuntu-devel, but hopefully it's mostly settled now.  I _think_ n-m will only manage stuff that is "auto" in /etc/network/interfaces
 168 [19:57] <kees> (or missing from interfaces)
 169 [19:57] <kees> [SLIDE 28]
 170 [19:57] <kees> if you're running devel on a remote machine, you're in for a world of hurt if it doesn't boot :)  be careful, get a console server, etc.
 171 [19:57] <kees> [SLIDE 29]
 172 [19:58] <kees> here's where to report bugs.  start with "ubuntu-bug $Package" first, and go from there.
 173 [19:58] <kees> [SLIDE 30]
 174 [19:58] <kees> that's it from me!
 175 [19:58] <cjohnston> < Guillo> QUESTION: What hapen with python-xml this librari don't install?
 176 [19:58] <kees> I've only got 2 minutes left, but I'll try to answer that
 177 [19:59] <kees> Guillo: I'd start my investigations by looking at what apt-cache has to say.  here's a quick demo:
 178 [19:59] <kees> $ apt-cache madison python-xml
 179 [19:59] <kees> and I see it's missing.
 180 [19:59] <kees> then I go to launchpad and look for it:
 181 [20:00] <kees> https://launchpad.net/ubuntu/+source/python-xml
 182 [20:00] <kees> I see it's missing since karmic
 183 [20:00] <kees> then I check Debian for it:
 184 [20:00] <kees> http://packages.qa.debian.org/p/python-xml.html
 185 [20:00] <kees> I see it was removed.
 186 [20:01] <kees> and then I read the Debian reasoning for it.
 187 [20:01] <kees> gotta go!  thanks everyone!