PbuilderHowto

Differences between revisions 100 and 101
Revision 100 as of 2008-01-30 15:54:13
Size: 21855
Editor: 24
Comment: remove superfluous touch
Revision 101 as of 2008-01-30 16:02:50
Size: 21834
Editor: 24
Comment: make use of etch and sid consistent instead of generic unstable/stable
Deletions are marked like this. Additions are marked like this.
Line 103: Line 103:
For new packages you need a chroot of the current unstable and under development of ubuntu. To do so, create a chroot of the current stable release and then do: For new packages you need a chroot of the current distribution under development of Ubuntu (hardy, at this writing). To do so, create a pbuilder chroot as usual and then do:
Line 271: Line 271:
There will be sometimes where running {{{sudo pbuilder create}}} will fail to create a base tarball. This normally occurs only when creating base tarballs for development versions of Ubuntu or Debian (for example, Debian unstable). These are development versions and it is best to wait and try at a later date to create a base tarball. It can be days however (sometimes weeks) until creating a base tarball will be successful for these development versions. In the meantime you could create a base tarball for an older version of Ubuntu or Debian and update the tarball to the development version instead. There will be sometimes where running {{{sudo pbuilder create}}} will fail to create a base tarball. This normally occurs only when creating base tarballs for development versions of Ubuntu or Debian (for example, Debian sid). These are development versions and it is best to wait and try at a later date to create a base tarball. It can be days however (sometimes weeks) until creating a base tarball will be successful for these development versions. In the meantime you could create a base tarball for an older version of Ubuntu or Debian and update the tarball to the development version instead.
Line 318: Line 318:
[unstable]

[stable]
[sid]

[etch]
Line 328: Line 328:
This conffile was modified from the documentation for mini-dinstall ({{{/usr/share/doc/mini-dinstall}}}). Under {{{architectures}}}, you can choose to only specify the architecture of your machine. The listing of {{{[unstable]}}}, {{{[gutsy]}}}, etc. is so that you will be able to upload packages using dput for Debian unstable or any Ubuntu version. If you are only building packages for one version, just list only that version. This conffile was modified from the documentation for mini-dinstall ({{{/usr/share/doc/mini-dinstall}}}). Under {{{architectures}}}, you can choose to only specify the architecture of your machine. The listing of {{{[sid]}}}, {{{[gutsy]}}}, etc. is so that you will be able to upload packages using dput for Debian sid or any Ubuntu version. If you are only building packages for one version, just list only that version.
Line 360: Line 360:
<distro-version> is either unstable, gutsy, etc. depending on what distro version you built the package for. In this directory, there should be a {{{Packages}}} and {{{Source}}} file along with the packages you uploaded. <distro-version> is either sid, gutsy, etc. depending on what distro version you built the package for. In this directory, there should be a {{{Packages}}} and {{{Source}}} file along with the packages you uploaded.
Line 414: Line 414:
            OTHERMIRROR="deb file:///usr/local/src/archive unstable/"             OTHERMIRROR="deb file:///usr/local/src/archive sid/"
Line 419: Line 419:
            OTHERMIRROR="deb file:///usr/local/src/archive stable/"             OTHERMIRROR="deb file:///usr/local/src/archive etch/"

Introduction

This document describes the steps to setup a chroot environment for building Ubuntu packages using pbuilder. It is recommended for any maintainer of Ubuntu or Debian packages, or anyone interested in building packages using Ubuntu.

You can build packages without pbuilder but you must already have all the build dependencies of the package that you are building installed on your running system.

If you use LVM and want to take advantage of using LVM snapshots to speed up the chroot creation/destruction, please see ["SbuildLVMHowto"] for an alternate approach.

The following assume you want to install edgy. If you need a dapper or feisty chroot, then exchange edgy with the appropriate distribution.

The First Steps

First install the required packages.

sudo apt-get install pbuilder debootstrap devscripts

NOTE: devscripts is not necessary to install along with pbuilder, however if you are serious about using pbuilder and creating and maintaining packages for Ubuntu or Debian, you will want to install devscripts.

Create a base tarball that will contain your chroot environment to build packages with.

sudo pbuilder create

To more closely mimic the environment of the official build machines, you can use the '--variant=buildd' option when creating the base tarball.

sudo pbuilder create --debootstrapopts --variant=buildd

Rebuilding a package

If everything went well, you should now have a tarball called base.tgz in your system under /var/cache/pbuilder.

To check if you can download source packages, you will need an uncommented deb-src line in /etc/apt/sources.list. It should look like:

    deb-src http://archive.ubuntu.com/ubuntu <ubuntu_version> main restricted universe multiverse

Replace <ubuntu_version> with the version of ubuntu you are using (i.e. feisty, edgy, or dapper). If the line above is there but not uncommented, uncomment it. Make sure your package index files are synchronized so that apt-get knows where to find the sources.

After you ensured the deb-src line is correct and uncommented, run:

    sudo apt-get update

We will build bc, a simple calculator, from source, let's download the debian source package from ubuntu repository:

    apt-get source bc

Now build the package using your chroot cleanroom you created with the command:

    sudo pbuilder build *.dsc

Once the packages are successfully built, the binary and source packages will be stored in /var/cache/pbuilder/result/

pdebuild

pdebuild is the pbuilder way of doing debuild. It comes along with the pbuilder package. pdebuild is essentially run the same way as debuild. Go into the source tree containing the debian directory and type:

pdebuild

See also the [http://penguin-soft.com/penguin/man/1/pdebuild.html pdebuild man page].

Downloading Source Packages Using dget

There will be times when a package you want or need to build is not in the repositories of the current version of Ubuntu you are running. Rather than entering deb-src lines in your sources.list configuration for other versions of Ubuntu you are not running, you can use dget instead to download a package. dget is included in the devscripts package, hence why you should have installed it.

To use dget, first locate the .dsc file of the package you are going to build. Good places to look are http://packages.ubuntu.com/ or http://packages.debian.org/ . Once you located the .dsc file. Copy the link address to it and then in a command prompt run:

    dget <link_address_to_foo.dsc>

NOTE: dget is not included in the devscripts package for Ubuntu dapper.

Signing Source Files

If you intend to upload your packages to ["REVU"], [http://mentors.debian.net/], or any other location, it is highly recommended you sign your source files (.dsc and .changes files). Both ["REVU"] and [http://mentors.debian.net/] enforce signed sources. The easiest way to do this after using pbuilder is by using debsign. Assuming your packages and source files were placed in /var/cache/pbuilder/result/, run;

sudo debsign /var/cache/pbuilder/result/*.changes

debsign is also a part of the package devscripts.

Updating a chroot

It is recommended you update your chroot daily before each build, to do this task use the command below:

    sudo pbuilder update

Updating a chroot to another version of ubuntu

For new packages you need a chroot of the current distribution under development of Ubuntu (hardy, at this writing). To do so, create a pbuilder chroot as usual and then do:

    sudo pbuilder update --distribution DIST-NAME --override-config

It will update the chroot to the DIST-NAME version of ubuntu and will override the configuration files to make it effective. Then you can start building packages for the next version of ubuntu.

Speeding up build-dependency calculation

If you use feisty you can use a new build-dependency resolver based on gdebi that should be significantly faster. Change ~/.pbuilderrc to add this line:

    PBUILDERSATISFYDEPENDSCMD="/usr/lib/pbuilder/pbuilder-satisfydepends-gdebi"

Problems with pbuilder-satisfydepends-gdebi and local repositories

If you are using a local overlay repository (a very advanced use of pbuilder usually used for backport archives), you will likely want to use a modified version of 'pbuilder-satisfydepends-gdebi'. First, copy the dependency resolver over, and mark it as executable.

   cp /usr/lib/pbuilder/pbuilder-satisfydepends-gdebi ~/.pbuilder-satisfydepends-gdebi
   chmod +x ~/.pbuilder-satisfydepends-gdebi

Change line 80 of '$HOME/.pbuilder-satisfydepends-gdebi' to ignore warnings from local packages that aren't in the Ubuntu keyring.

Change:

   $CHROOTEXEC /usr/bin/apt-get install -y $INSTALL

To this:

   $CHROOTEXEC /usr/bin/apt-get install -y --force-yes $INSTALL

Then change ~/.pbuilderrc to use your local modified dependency resolver script instead of the one which comes with pbuilder by default:

   PBUILDERSATISFYDEPENDSCMD="~/.pbuilder-satisfydepends-gdebi"

Integration with ccache

Using ccache with pbuilder is easy, simply add the following to ~/.pbuilderrc (/etc/pbuilder/pbuilderrc is an alternative):

# ccache
sudo mkdir -p /var/cache/pbuilder/ccache
sudo chmod a+w /var/cache/pbuilder/ccache
export CCACHE_DIR="/var/cache/pbuilder/ccache"
export PATH="/usr/lib/ccache:${PATH}"
EXTRAPACKAGES=ccache
BINDMOUNTS="${CCACHE_DIR}"

Now pbuilder will automatically cache compiler output between multiple builds of the same software.

Universe support

If you want to rebuild a package from universe or build a new one that needs a package out of main you will need to add this to ~/.pbuilderrc:

COMPONENTS="main restricted universe multiverse"

if [ "$DIST" == "etch" ]; then
    echo "Using a Debian pbuilder environment because DIST is $DIST"
    COMPONENTS="main contrib non-free"
fi

After adding the new sources you need to update the chroot, so that it picks up the new apt lines:

sudo pbuilder update --override-config

Building an i386 pbuilder on AMD64

One of the things a pbuilder is useful for is building i386 packages on an AMD64 machine. You can create an i386 chroot with the command:

sudo pbuilder create --debootstrapopts --arch --debootstrapopts i386

Multiple pbuilders

When you work with packages, you often want to have multiple pbuilders on your machine, if for example you want to backport to Dapper while developing on the development version of Ubuntu (currently Hardy). Or maybe you want to [https://wiki.ubuntu.com/ContributingToDebian build your packages for Debian] and have them merged back to Ubuntu. The next sections will provide some information to get you started with using multiple pbuilders.

Update debootstrap

It is recommended to use an updated version of debootstrap from the backports repository of whatever version of Ubuntu you are using. You can either activate the backports repository in your sources.list configuration or download the source package using dget, build the packages, and install them. The updated version will have support for newer distributions, such as the development branch of Ubuntu.

Change base.tgz Location

The only thing required to use pbuilder with multiple distributions is an alternate location to store the gzipped tarball that contains the pbuilder environment. On the command line, this can be specified with the 'basetgz' option. However, it is tedious to specify the full path every time pbuilder is run, so it is convenient to place a snippet in ~/.pbuilderrc to automate this:

# The distribution should be correct in your changelog file.
# Uncomment this line to use the one listed there:
#DIST=`dpkg-parsechangelog | awk '/^Distribution: / {print $2}'`

if [ -n "${DIST}" ] ; then
    NAME="$DIST"
    if [ -n "${ARCH}" ] ; then
        NAME="$NAME-$ARCH"
        DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
        export DEBIAN_BUILDARCH="$ARCH"
    fi
    BASETGZ="`dirname $BASETGZ`/$NAME-base.tgz"
    DISTRIBUTION="$DIST"
    BUILDRESULT="/var/cache/pbuilder/$NAME/result/"
    APTCACHE="/var/cache/pbuilder/$NAME/aptcache/"

    case "$DIST" in
        hardy|gutsy)   # ubuntu specific
            MIRRORSITE="http://mirrors.kernel.org/ubuntu/"
            COMPONENTS="main restricted universe multiverse"
            ;;
        sid|etch)      # debian specific
            MIRRORSITE="http://mirrors.kernel.org/debian/"
            COMPONENTS="main contrib non-free"
            ;;
        "*")
            echo "Unknown distribution: $DIST"
            exit 1
            ;;
    esac
fi

Now, if the user specifies a value for DIST when running pbuilder, such as gutsy, the tarball location will be changed. The line that sets the DISTRIBUTION only takes effect during the creation of a new base tarball, or if the --override-config option is given, where it specifies the distribution to use for the new base tarball. Setting BUILDRESULT or APTCACHE is optional, but possibly helpful.

If ARCH is also specified when running pbuilder, pbuilder will be set to create an environment to build packages for the architecture specified in ARCH.

We can now create and use alternate tarballs, as in the following examples:

# Create a base environment for Ubuntu gutsy
DIST=gutsy sudo pbuilder create

# Create a base environment for Debian sid
DIST=sid sudo pbuilder

# Create a base environment for Ubuntu gutsy under
# the i386 architecture
DIST=gutsy ARCH=i386 sudo pbuilder create

# Update a base environment for Ubuntu gutsy
DIST=gutsy sudo pbuilder update

# Build a package using Ubuntu gutsy as the base
# environment
DIST=gutsy pdebuild

# Build a package using Ubuntu gutsy as the base
# environment under the i386 architecture
DIST=gutsy ARCH=i386 pdebuild

Using the 'othermirror' option

It is possible to specify more mirrors than the one specified for MIRRORSITE. There's two ways to do this, specifiying the option '--othermirror "<sources.list deb line>"' when running pbuilder, or using the variable 'OTHERMIRROR="<sources.list deb line>"' in the pbuilderrc configuration file. Just replace "<sources.list deb line>" with a deb line in the same format as would be found under /etc/apt/sources.list.

Troubleshooting "pbuilder create"

There will be sometimes where running sudo pbuilder create will fail to create a base tarball. This normally occurs only when creating base tarballs for development versions of Ubuntu or Debian (for example, Debian sid). These are development versions and it is best to wait and try at a later date to create a base tarball. It can be days however (sometimes weeks) until creating a base tarball will be successful for these development versions. In the meantime you could create a base tarball for an older version of Ubuntu or Debian and update the tarball to the development version instead.

For example, to create an etch base tarball and update to sid, start with:

DIST=etch sudo pbuilder create

Then to update, run:

DIST=etch sudo pbuilder update --override-config \
--othermirror "deb http://http.us.debian.org/debian sid main"

What will happen is that the base packages for Etch will be downloaded, installed, and configured first. Then apt-get update will be performed for "etch main" and "sid main". When the update is run, all packages in need of upgrading will be upgraded and thus, the base packages for Sid will be installed. After pbuilder is done, a base tarball will have been created but named for Etch (etch-base.tgz). All the base packages will be copied into the aptcache directory in /var/cache/pbuilder/etch. From here you can copy the aptcache directory over to the sid directory and make a copy of etch-base.tgz and call it sid-base.tgz.

sudo mkdir /var/cache/pbuilder/sid
sudo cp /var/cache/pbuilder/etch/aptcache /var/cache/pbuilder/sid/aptcache
sudo cp /var/cache/pbuilder/etch-base.tgz /var/cache/pbuilder/sid-base.tgz

It is better of course to create a base tarball directly. Check often to see if a base tarball can be created directly.

Building With Local Packages

Sometimes a package you intend to build will depend on a recent version of a package that isn't found in the Ubuntu or Debian archives. Or your package depends on another package that isn't in the Ubuntu or Debian archives at all. For these type of cases, you could create the packages and upload them to your local filesystem as you wait for a package to be uploaded to the archives. The next sections will cover setting up a miniature repository on your system for use with pbuilder.

Setting up

First, install the required packages.

sudo apt-get install dput mini-dinstall

Create a file call .mini-dinstall.conf under your home directorywith the following contents:

[DEFAULT]
architectures = all, i386, amd64, powerpc
archivedir = /usr/local/src/archive/
use_dnotify = 0
verify_sigs = 0
extra_keyrings = ~/.gnupg/pubring.gpg
mail_on_success = 0
archive_style = flat
poll_time = 10
mail_log_level = NONE

[sid]

[etch]

[hardy]

[gutsy]

This conffile was modified from the documentation for mini-dinstall (/usr/share/doc/mini-dinstall). Under architectures, you can choose to only specify the architecture of your machine. The listing of [sid], [gutsy], etc. is so that you will be able to upload packages using dput for Debian sid or any Ubuntu version. If you are only building packages for one version, just list only that version.

Edit the [local] stanza under /etc/dput.cf to look like:

[local]
method = local
incoming = /usr/local/src/archive/mini-dinstall/incoming
allow_non-us_software = 1
run_dinstall = 0
post_upload_command = mini-dinstall --batch

Now create the "incoming" directory for your local repository.

sudo mkdir -p /usr/local/src/archive/mini-dinstall/incoming

Uploading to Local Filesystem

Once you build a package, you can upload it to your local filesystem. Run:

sudo dput local foo*.changes

With the configuration above, it should end up in

/usr/local/src/archive/<distro-version>

<distro-version> is either sid, gutsy, etc. depending on what distro version you built the package for. In this directory, there should be a Packages and Source file along with the packages you uploaded.

Updating Using the Local Repository

Now do an update of pbuilder. You will need to run pbuilder with the 'override-config' first before you base environments will use the new "deb" location. You will also need to use the 'bindmounts' option to mount the directory with your local repository to the base environment and you will need to use the 'othermirror' option to specify a "deb" line for the local repository. For example.

# This updates a base environment of Ubuntu gutsy to
# use a local repository at '/usr/local/src/archive/'
DIST=gutsy sudo pbuilder update \
--bindmounts /usr/local/src/archive/ --override-config \
--othermirror "deb file:///usr/local/src/archive/ gutsy/"

# This updates a base environment of Debian sid to
# use a local repository at '/usr/local/src/archive/'
# The 'mirror' option also needs to be specified here
DIST=sid sudo pbuilder update \
--bindmounts /usr/local/src/archive/ --override-config \
--mirror "http://http.us.debian.org/debian/" \
--othermirror "deb file:///usr/local/src/archive/ gutsy/"

Then, you should rewrite your pbuilderrc configuration file to place a snippet to look like this.

# The distribution should be correct in your changelog file.
# Uncomment this line to use the one listed there:
#DIST=`dpkg-parsechangelog | awk '/^Distribution: / {print $2}'`

if [ -n "${DIST}" ] ; then
    NAME="$DIST"
    if [ -n "${ARCH}" ] ; then
        NAME="$NAME-$ARCH"
        DEBOOTSTRAPOPTS=("--arch" "$ARCH" "${DEBOOTSTRAPOPTS[@]}")
        export DEBIAN_BUILDARCH="$ARCH"
    fi
    BASETGZ="`dirname $BASETGZ`/$NAME-base.tgz"
    DISTRIBUTION="$DIST"
    BUILDRESULT="/var/cache/pbuilder/$NAME/result/"
    APTCACHE="/var/cache/pbuilder/$NAME/aptcache/"
    BINDMOUNTS="/usr/local/src/archive"

    case "$DIST" in
        hardy)          # ubuntu specific
            MIRRORSITE="http://us.archive.ubuntu.com/ubuntu"
            COMPONENTS="main restricted universe multiverse"
            OTHERMIRROR="deb file:///usr/local/src/archive hardy/"
            ;;
        gutsy)          # ubuntu specific
            MIRRORSITE="http://us.archive.ubuntu.com/ubuntu"
            COMPONENTS="main restricted universe multiverse"
            OTHERMIRROR="deb file:///usr/local/src/archive gutsy/"
            ;;
        sid)            # debian specific
            MIRRORSITE="http://http.us.debian.org/debian/"
            COMPONENTS="main contrib non-free"
            OTHERMIRROR="deb file:///usr/local/src/archive sid/"
            ;;
        etch)           # debian specific
            MIRRORSITE="http://http.us.debian.org/debian/"
            COMPONENTS="main contrib non-free"
            OTHERMIRROR="deb file:///usr/local/src/archive etch/"
            ;;
        "*")
            echo "Unknown distribution: $DIST"
            exit 1
            ;;
    esac
fi

After updating each base environment to use your local repository, you can use the pbuilder/pdebuild commands as before.

Installing Pbuilder on Warty, Hoary or Breezy

Install pbuilder package and debootstrap.

Add this to ~/.pbuilderrc:

    MIRRORSITE=http://archive.ubuntu.com/ubuntu
    DISTRIBUTION=edgy

Since pbuilder will complain about unauthenticated packages, you have to allow unauthenticated packages. You can make it like so:

Add this to ~/.pbuilderrc:

    APTCONFDIR="/etc/pbuilder/apt.config/"

Now you'll copy needed files of /etc/apt to it and

    sudo -s
    mkdir /etc/pbuilder/apt.config/
    cp -ar /etc/apt/* /etc/pbuilder/apt.config/
    echo "APT::Get::AllowUnauthenticated 1;" >> /etc/pbuilder/apt.config/apt.conf.d/allow-unauthenticated
    exit

NOTE: Make sure to remove 99update-notifier from /etc/pbuilder/apt.config/apt.conf.d/ if you have copied it in the last step. It can potentially break future updates to your pbuilder chroot, as update-notifier won't normally be installed.

Save the file and run the command below to build your Edgy chroot:

    sudo pbuilder create --distribution edgy

Running a Shell When Build Fails (Intro to Hook Scripts)

This is done using a hook script. First we need to create a directory to home your pbuilder hooks. This can be anywhere but we'll use a directory called .pbuilder-hooks in your home directory for this example:

    mkdir ~/.pbuilder-hooks

Now we need to tell pbuilder to use this directory as its hooks directory. Edit ~/.pbuilderrc and add the following line, replacing <username> with your username:

    HOOKDIR=/home/<username>/.pbuilder-hooks

Then we need to add a hook script to be called by pbuilder when the build fails. The script needs to be placed in the HOOKDIR and made executable. The name of the hook script is also important. See the pbuilder man page for details. As you can see, for this example, we need to name the script C<digit><digit><whatever-else-you-want> so let's call it C10shell. Edit the new file ~/.pbuilder-hooks/C10shell and add the following:

# invoke shell if build fails.

apt-get install -y --force-yes vim less bash
cd /tmp/buildd/*/debian/..
/bin/bash < /dev/tty > /dev/tty 2> /dev/tty

This script must be made world executable for pbuilder to execute it, so now we run:

    chmod a+x ~/.pbuilder-hooks/C10shell

And you're all set.

See also

pbuilder User's Manual: http://www.netfort.gr.jp/~dancer/software/pbuilder-doc/pbuilder-doc.html

See the script pbuilder-dist in package ubuntu-dev-tools for an alternative way to use pbuilder with many different Ubuntu versions.


["CategoryMOTU"]

PbuilderHowto (last edited 2020-01-28 13:36:36 by paelzer)