Ubuntu Wiki

   1 [16:28] <tumbleweed> I was still writing some notes, but no reason not to start a couple of minutes early :)
   2 [16:28] <tumbleweed> Hi everyone,
   3 [16:28] <tumbleweed> Some of you may have been at the session I took yesterday afternoon, on working with Debian
   4 [16:28] <tumbleweed> For everyone else, hi, I'm Stefano Rivera, an Ubuntu MOTU and Debian Developer
   5 [16:28] <tumbleweed> I'm here to talk to you about building packages locally with pbuilder
   6 [16:28] <tumbleweed> If you followed Daniel Holbach's introduction on Tuseday, you'll have already set up pbuilder, using pbuilder-dist. https://wiki.ubuntu.com/MeetingLogs/devweek1201/DevEnvironmentSetup
   7 [16:28] <tumbleweed> If you haven't, install ubuntu-dev-tools and pbuilder right now, and run "pbuilder-dist precise create" (it'll take quite a while)
   8 [16:29] <tumbleweed> Maybe you'll be done by the end of the session... :)
   9 [16:29] <tumbleweed> As you can imagine from its name, pbuilder is a tool for building Debian packages
  10 [16:30] <tumbleweed> (Ubuntu packages aren't any different)
  11 [16:30] <tumbleweed> If you haven't used pbuilder before, you make wonder what the point is: Anyone can just build a source package by extracting it and running debuild, right?
  12 [16:31] <tumbleweed> Well, you need all the build dependencancies installed, so you're machine will quickly end up with lots of packages that you don't really need installed.
  13 [16:31] <tumbleweed> Your machine will have packages installed in it that the official build machines don't
  14 [16:31] <tumbleweed> All packages in the main archive are built in restricted build environments on buildds (build machines)
  15 [16:32] <tumbleweed> so, just building locally with debuild may give different results
  16 [16:32] <tumbleweed> (with well behaved packages it doesn't, but we probably spend more time on the badly behaving ones, right? :)
  17 [16:32] <tumbleweed> The buildds run a program called sbuild to do the builds. It installs the build dependencies in a minimal chroot, builds the package, and then throws away the chroot
  18 [16:32] <tumbleweed> We'd like to have a similar environment for testing builds
  19 [16:33] <tumbleweed> Something we can use to find likley build failures before we upload
  20 [16:33] <tumbleweed> (you do test before uploading, right?)
  21 [16:33] <tumbleweed> and to help us debug failures
  22 [16:33] <tumbleweed> Launchpad has PPAs, using the same build systems an the primary Ubuntu archives, why do you need to build things locally?
  23 [16:33] <tumbleweed> Well, there are a few reasons:
  24 [16:33] <tumbleweed> Launchpad's PPA builders can get rather backed up, sometimes the queue gets as long as a few days (currently they are quite short, though: https://launchpad.net/builders )
  25 [16:34] <tumbleweed> Your machine is probably faster than some of the buildds (at least mine is :P )
  26 [16:34] <tumbleweed> You don't have to bump the version every time you want to try a build
  27 [16:34] <tumbleweed> You can log into the build environment and debug things, when they go wrong
  28 [16:34] <tumbleweed> ^ That one is the killer feature
  29 [16:34] <tumbleweed> So, we want to set up a reproducable minimal build environment locally, that we can use for testing package builds and debugging them
  30 [16:34] <tumbleweed> You could use sbuild, like the buildds, or you could use pbuilder, which is simpler and easier to customise.
  31 [16:35] <tumbleweed> ere's also cowbuilder, which is a "faster pbuilder", but I won't bother going into that, if you're reaching the limits of pbuilder, I suggest also looking at sbuild
  32 [16:35] <tumbleweed> err "There's"
  33 [16:36] <tumbleweed> Personally, I build everything with sbuild, but I'm here to talk about pbuilder, so that's what I'm doing :P I used to use it, there's nothing wrong with it
  34 [16:36] <tumbleweed> now seems to be a good time to see if I've actually got an audience
  35 [16:36] <tumbleweed> anyone currently building a pbuilder chroot?
  36 [16:36] <tumbleweed> anyone run into trouble already?
  37 [16:37] <tumbleweed> So, details:
  38 [16:37] <tumbleweed> What pbuilder does:
  39 [16:37] <tumbleweed> When you create a chroot, it debootstraps a clean install into a directory (debootstrap is a very low level tool for minimal Debian/Ubuntu installs)
  40 [16:37] <tumbleweed> It then packs up that directory into a tarball.
  41 [16:37] <tumbleweed> Whenever you do a build, it upacks that tarball, chroots into the unpack directory, and does the build.
  42 [16:37] <tumbleweed> The directory can be deleted afterwards
  43 [16:38] <tumbleweed> (It's worth noting that sbuild used to be far more complicated, using LVM, but these days it supports overlay filesystems, so it's totally reasonable to set it up locally)
  44 [16:38] <tumbleweed> So, how come I'm talking about pbuilder when I told you to run "pbuilder-dist" ?
  45 [16:38] <tumbleweed> Well, it turns out that pbuilder is very flexible, but doesn't come with all the features you'd want, out the box
  46 [16:38] <tumbleweed> It assumes that you are building everything for a single distribution and release (which makes some sense in Debian, but less so in Ubuntu)
  47 [16:38] <tumbleweed> So, we have a wrapper around pbuilder in ubuntu called pbuilder-dist
  48 [16:39] <tumbleweed> All it does is call pbuilder with a bunch of command line options (you can see that with a ps x)
  49 [16:39] <tumbleweed> It also puts the chroot tarbals and built files into a friendly ~/pbuilder/ rather than /var/cache/pbuilder/
  50 [16:39] <tumbleweed> before pbuilder-dist, most people used to use complicated pbuilder configuratino files that did all of that for them
  51 [16:39] <tumbleweed> figuring out what release to build for, from environment variables
  52 [16:40] <tumbleweed> e.g. https://wiki.ubuntu.com/PbuilderHowto#Multiple_pbuilders
  53 [16:40] <tumbleweed> pbuilder-dist makes it a lot quciker for you to get started
  54 [16:40] <tumbleweed> Right, so, if you have a working pbuilder, you can test building something in it
  55 [16:41] <tumbleweed> grab a package that'll build quickly, here's one I can think of, beautifulsoup:
  56 [16:41] <tumbleweed> $ pull-lp-source -d beautifulsoup
  57 [16:41] <tumbleweed> $ pbuilder-dist precise beautifulsoup_3.2.0-2build1.dsc
  58 [16:42] <tumbleweed> and it should build...
  59 [16:42] <ClassBot> coolbhavi asked: How to get pbuilder log files?
  60 [16:42] <tumbleweed> pbuilder will store a log file next to the build output
  61 [16:43] <tumbleweed> so, next to the debs, you should see a file called beautifulsoup_3.2.0-2build1_amd64.build
  62 [16:43] <tumbleweed> (or i386, if you use i386)
  63 [16:44] <tumbleweed> Next up: pbuilder maintainance
  64 [16:44] <tumbleweed> it's not very useful to build in last week's development environment, Ubuntu development releases move fast
  65 [16:44] <tumbleweed> you always want to build with current packages
  66 [16:45] <tumbleweed> so, every now and then (say, when you start working on something for the first time in the day, run pbuilder-dist update precise)
  67 [16:45] <tumbleweed> (you may set up a cronjob for that if you do this a lot)
  68 [16:45] <ClassBot> coolbhavi asked: I heard about pbuilder and pdebuild whats the difference between them?
  69 [16:46] <tumbleweed> right, pdebuild is like "debuild" for pbuilder
  70 [16:46] <tumbleweed> you can run it inside an extracted source package, and it'll build the source package, and pass it to pbuilder
  71 [16:46] <tumbleweed> unfortunetly, we don't have a pdebuild for pbuilder-dist (there's a bug requesting one)
  72 [16:46] <tumbleweed> so it's not very useful for pbuilder-dist users
  73 [16:47] <tumbleweed> OK, onwards!
  74 [16:48] <tumbleweed> right, so you updated your pbuilder 3 hours ago, and it's already clearly out of date, because you are seeing 404 errors during the dependancy installation phase
  75 [16:48] <tumbleweed> it'd make a lot more sense to make it update itself at the start of the build
  76 [16:48] <tumbleweed> fortunately, pbuilder comes with a "hook script" feature
  77 [16:48] <tumbleweed> you can make it run programs at certain times
  78 [16:48] <tumbleweed> (you can read all the gorey details in the pbuilder manpage)
  79 [16:48] <tumbleweed> there are a few nice example scripts in /usr/share/doc/pbuilder/examples/ that I want to bring to your attention
  80 [16:50] <tumbleweed> hrm, I swear there used to be a D90update script there, but I don't see one
  81 [16:50] <ClassBot> There are 10 minutes remaining in the current session.
  82 [16:50] <tumbleweed> anyway, I have my own, which just runs "/usr/bin/apt-get update"
  83 [16:50] <tumbleweed> handy for making sure the package lists are up to date before building
  84 [16:51] <tumbleweed> there are also examples (debc and lintian) for outputting useful information at the end of the build
  85 [16:51] <tumbleweed> debc shows you what's in the built packages
  86 [16:51] <tumbleweed> lintian does some cursory checks on the packages
  87 [16:51] <tumbleweed> (all of this ends up in your build log, and is very handy for debugging)
  88 [16:52] <tumbleweed> the dpkg-i script tries to install and remove the packages that were built
  89 [16:52] <tumbleweed> this can pick up problems in the maintainer scripts
  90 [16:52] <tumbleweed> (you can also do this separately with piuparts)
  91 [16:52] <tumbleweed> and by far the most useful one, C10shell
  92 [16:52] <tumbleweed> this is awesome
  93 [16:53] <tumbleweed> when a build fails, it dumps you in a shell, inside the build chroot
  94 [16:53] <tumbleweed> so you can play around, and figure out what went wrong and how to fix it
  95 [16:53] <tumbleweed> I highly recommend enabling it
  96 [16:53] <tumbleweed> how do you use these? Well you create a directory in your home directory, traditionally ~/.pbuilder-hooks
  97 [16:54] <tumbleweed> then you put HOOKDIR=$HOME/.pbuilder-hooks in your .pbuilderrc
  98 [16:54] <ClassBot> jincreator asked: You mentioned "pbuilder-dist update precise" but I think it is "pbuilder-dist precise update"...
  99 [16:54] <tumbleweed> jincreator: that may be true, I'm rusty
 100 [16:54] <tumbleweed> I know it does offer some leewaywith getting arguments in the wrong order
 101 [16:55] <ClassBot> There are 5 minutes remaining in the current session.
 102 [16:55] <tumbleweed> another pretty cool thing you can do with pbuilder is to build on a different architecture
 103 [16:55] <tumbleweed> if instead of saying "precise" you say "precise-i386", it'll create an i386 chroot on your amd64 system. Handy
 104 [16:56] <tumbleweed> you can even build armel packages on i386/amd64, through emulation with qemu-user-static
 105 [16:56] <tumbleweed> It'll sometimes not work, but it mostly does
 106 [16:56] <tumbleweed> (I'm talking about qemu there, sometimes it segfaults unexpectadly)
 107 [16:57] <tumbleweed> final questions?
 108 [16:57] <tumbleweed> Sorry that was crazy fast, 30 min slots are short...
 109 [16:58] <tumbleweed> There are lots of people who know pbuilder, cowbuilder and sbuild backwards, who hang out in #ubuntu-motu, feel free to ask us questions any time
 110 [16:59] <ClassBot> alucardni asked: how can I set pbuilder to build the package twice?
 111 [16:59] <tumbleweed> You just say --twice
 112 [16:59] <tumbleweed> That's really handy for finding packages that don't clean correctly
 113 [16:59] <tumbleweed> that is, their clean rule doesn't cleanup everything that the build created / modified
 114 [17:00] <tumbleweed> OK, I'm done, thanks for listening