UsingDevelopmentReleases
Getting Started |
In order to work on and actively improve Ubuntu, fixes need to be built, tested and integrated into the current development release. (Fixes for stable release follow their own policy.)
On this page we'll discuss measures to safely run and test using the active development release.
Testing and Working
Before you Get Started
Ubuntu Live Presentation "The Bleeding Edge" by Kees Cook and Colin Watson
Getting Started
TODO: This section needs more detail and more work.
Welcome to the wonderful world of Ubuntu testing. This guide will help you avoid common pitfalls of running an Ubuntu developmental release. Some guidelines:
Check the changelogs on the changes list.
The Archive Status Page to see which packages are broken and if they're important to you.
- sudo apt-get update
- sudo apt-get upgrade
- Held Back packages:
- Examples go here
- Never do a partial upgrade, things are held back for a reason.
- Wait for the archive to update
- apt-get dist-upgrade does what you tell it to do, if the archive is in flux then it will remove things to keep your computer consistent. Usually you do not want this, so don't do it. If a package is held up for a long time (a few days), then try to apt-get install it, it might be held back because it depends on new packages. If you have a bunch held back just go through them individually and see if anything wants to be removed.
- Sometimes it's obvious that it wants to remove something and replace it with something newer (libfoo1 is going to be REMOVED, replaced by libfoo2)
- Update manager
- Don't just click partial upgrade, odds are eventually it will break.
- Held Back packages:
Getting Help
General
"Living on the Edge" can sometimes involve debugging problems, asking for help and filing bug reports if you encounter problems.
IRC: #ubuntu+1 on irc.freenode.net for support and to find out if a certain problem is already known
please use #ubuntu-devel only if helping with development
please use #ubuntu-bugs if helping report/triage bugs
If you use the Ubuntu Forums, ask for help in the development forum.
You can also ask a question in the Launchpad Answers tracker.
If you encounter serious problems, that looks like maintainers should know about, consider reporting a bug
add as much information to the bug reports as possible: DebuggingProcedures
If you're on the ubuntu-devel-discuss mailing list, you can ask for input on an issue or a possible fix for it there.
Specific
Mirror Problems
- checksum failures (development branch changes hourly; sometimes mirrors get skewed)
- unauthenticated package warnings
- don't install unauthenticated packages; this warning is for your protection (it could be a real attack))
- next mirror push may resolve the problems, so wait an hour and try again
- try temporarily using archive.ubuntu.com
- remember to switch back! it's slow and expensive
Uninstallable packages
- incorrect dependencies
- file-level conflicts ("trying to overwrite '/bar/baz', which is also in package “foo")
- broken maintainer scripts
- note that apt will fail if the system is too inconsistent, and you may have to fall back to using dpkg directly
Unpack failures
- typically haven't done much to your system
- if due to file-level conflict, may be fixable using:
- sudo dpkg --force-overwrite -i /var/cache/apt/archives/foo.deb
- almost always indicates a missing Replaces field
- bugs in pre-installation scripts generally unfixable without repacking .deb
- sometimes pre-removal script of old package may be broken; may require editing by hand
- see /var/lib/dpkg/info/foo.prerm
Configure Options
- package's files are unpacked, but post-installation script failed
- sometimes a retry is enough
- sudo dpkg --configure -a
- sometimes post-installation script is broken; may require editing by hand
- see /var/lib/dpkg/info/foo.postinst
- update-alternatives and dpkg-divert are often used and sometimes used wrongly; see the documentation if need be
Package system debugging
- dpkg itself only gives you an exit code, which isn't very useful
- read back through the output for the earliest error that caused dpkg to fail, and fix that
- there is a log in /var/log/apt/term.log that contains the dpkg output when the scripts were run
- when reporting a bug, quote the full output, not just the end
- most maintainer scripts are shell; to get a full execution trace, edit them in /var/lib/dpkg/info/, put 'set -x' on the second line, and retry
- if debconf goes wrong (often indicated by exit codes that are multiples of 10), get a debconf trace:
- export DEBCONF_DEBUG=developer
exit codes 10 and 20 are usually bad arguments to db_something, while 30 is often just a missing “|| true” after db_input or db_go
- add verbose flags to commands
strace (sledgehammer tracing tool, output can be hard to read)
sudo strace -f dpkg-command... 2>/tmp/foo.out
- debug apt-get dependency problems; output can be hard to read
- sudo apt-get -o Debug::pkgProblemResolver=true ...
- occasionally dpkg itself goes wrong; see help
- dpkg –debug=help
- dpkg debug options do not help with maintainer script problems!
Incorrect removals
- always check removal list carefully when performing a dist-upgrade
- update-manager has a list of known-good removals, but this may not be up to date
- "Following essential packages will be removed"
- never say yes unless you are well beyond needing this talk
- packages that have been removed from the archive are usually fair game to remove
- check package states with apt-cache policy PACKAGE
Administrative error
- forcibly installed package that causes problems
- sudo dpkg --remove foo; sudo apt-get -f install
- packaging system files were modified by hand
- reinstall packages and next time use dpkg-divert or dpkg-statoverride as necessary
- packages from third-party archives
- we don't deliberately break them, but sometimes it's unavoidable or unnoticed
- consider removing them to make the upgrade finish, and reinstall later
- report bugs to third party in question
Crashes during upgrades
- try to resist the urge to pull the power cord during upgrades, but if you must...
- sudo dpkg --configure -a
- sudo apt-get -f install
- "Package is in a very bad inconsistent state"
- dpkg --unpack /var/cache/apt/archives/foo.deb
- dpkg --configure -a
- /var/lib/dpkg/status is critically important
- copies in /var/lib/dpkg/status-old and /var/backups/
- /var/lib/dpkg/available corrupt
- sudo dselect update
- files filled with zero bytes
XFS does this if you're unlucky; get a UPS
- corrupt files belonging to packages
- dpkg -S /path/to/file
- sudo apt-get install --reinstall PACKAGE
- /var/cache/apt files can be removed if corrupt
- debconf database files can sort of be removed if corrupt, but you may have to reinstall packages and re-answer questions afterwards
- sudo dpkg-reconfigure -plow PACKAGE
Links: DebuggingUpdateManager
Hardware-specific breakage
- most important area to report bugs
- Not everyone has your hardware, so it can be a challenge to test all combinations.
Kernel breakage
- new kernel (or initramfs) breaks on boot
- always keep at least one old known-good kernel
- kernel ABI changes frequently in development branches so you will generally be able to boot an old one
- linux-image-2.6.22-8-generic 2.6.22-8.18
- kernel seems to work, but oopses later
- be cautious (is the filesystem oopsing?)
- report a bug and consider reverting to an older kernel
- include full oops log in /var/log/kern.log and/or dmesg
Links: KernelTeamBugPolicies, DebuggingKernelOops, DebuggingKernelSuspend
X.org breakage
- log in at console (Ctrl-Alt-F1)
- can generally fall back to vesa or vga driver
edit /etc/X11/xorg.conf, e.g.:
Section “Device” Driver “vesa” EndSection
- restart X.org
- sudo /etc/init.d/gdm restart
- should be less of a problem in future (bullet-proof-x), with automatic vesa/vga fallback
Links: X/Debugging
Network breakage
- network-manager takes down the network interface you were using
- interaction with /etc/network/interfaces is hard
- may require manual prodding (ifconfig, ifup/ifdown)
- if want you, you can stop network-manager
- sudo /etc/dbus-1/event.d/26NetworkManagerDispatcher stop
- sudo /etc/dbus-1/event.d/25NetworkManager stop
Links: DebuggingNetworkManager
Unpredictable bugs
- almost anything can happen in theory
- keep a login session open in case authentication breaks
- check /var/log/auth.log for authentication problems
- may need to reboot in recovery mode to fix
- in a pinch, (e)dit the grub menu item and use this for the kernel command-line arguments instead of “ro”:
- rw init=/bin/sh
- in a pinch, (e)dit the grub menu item and use this for the kernel command-line arguments instead of “ro”:
- report any library crashes or compilation breakage
Links: DebuggingProcedures, Bugs/FindRightPackage, DebuggingProgramCrash, DebuggingSystemCrash
Remote upgrades
<broken record>not a good idea with development branches</broken record>
- keep an ssh session open
- make sure you can still log in before closing it
- may still lose the battle if network interfaces go away
if you need to reboot remotely, invest in a remote console server
Without Any Tools
Devel release as main work environment |
|
Howto |
|
quick Howto |
sudo vi /etc/apt/sources.list ; sudo apt-get update && sudo apt-get dist-upgrade |
Advantages |
- tests the devel release on real HW |
Disadvantages |
- will have bugs |
separate partition with Ubuntu development release |
|
Howto |
|
quick Howto |
|
Advantages |
- tests the devel release on real HW |
Disadvantages |
- rebooting into devel partition takes time |
separate machine with Ubuntu development release |
|
Howto |
|
quick Howto |
|
Advantages |
- tests the devel release on real HW |
Disadvantages |
- requires more electricity and desk space |
use the LiveCD |
|
Howto |
|
quick Howto |
download iso from http://cdimage.ubuntu.com/daily-live/current/ - burn it - boot it |
Advantages |
- tests the devel release on real HW |
Disadvantages |
- rebooting into devel partition takes time |
Using Tools
chroot |
|
Howto |
|
quick Howto |
sudo apt-get install debootstrap; sudo debootstrap <development release> <directory> |
Advantages |
- easy setup |
Disadvantages |
- testing graphical applications isn't straight-forward |
kvm |
|
Howto |
|
quick Howto |
sudo apt-get install kvm virt-manager libvirt-bin |
Advantages |
- Feels like a real machine |
Disadvantages |
- Does not test the devel release on different hardware |
virtualbox |
|
Howto |
|
quick Howto |
sudo apt-get install virtualbox-ose virtualbox-ose-modules-uname -r |
Advantages |
- Feels like a real machine |
Disadvantages |
- Does not test the devel release on different hardware |
vmware |
|
Howto |
|
quick Howto |
|
Advantages |
- supports snapshots |
Disadvantages |
- not free software |
Building Packages
pbuilder |
|
Howto |
|
quick Howto |
sudo apt-get install pbuilder; sudo pbuilder create |
Advantages |
- easy setup |
Disadvantages |
- less useful for testing fixes |
using a PPA |
|
Howto |
|
quick Howto |
|
Advantages |
- good way to test if a package builds on various releases |
Disadvantages |
- just covers the 'build side' of testing |
sbuild |
|
Howto |
|
quick Howto |
sudo apt-get install ubuntu-dev-tools; mk-sbuild-lv <vg> <release> |
Advantages |
- fast |
Disadvantages |
- requires LVM and spare LVs |
Go back to UbuntuDevelopment.
CategoryUbuntuDevelopment