Ubuntu Wiki

Dev Week -- Kernel module packaging with DKMS -- Tue, Sep 2

<superm1> Hi everyone, my name is Mario Limonciello.  You may know me from things such as Mythbuntu & Ubuntu development.  I'm here today to talk to you a little bit about a project that I maintain called DKMS.
<superm1> I assume that most of the people here have heard about DKMS, so I'll try to be brief in my overview of it.  DKMS stands for Dynamic Kernel Module System (http://linux.dell.com/dkms/).
<superm1> It's original purpose was to allow the ability to ship updated versions of kernel modules without necessarily having to rebuild the kernel or needing to rebuild the kernel modules themselves again every time the kernel was upgraded.  Timing of new kernel releases and seeing patches get upstream don't always mesh well with release schedules, so it was intended to be an interim solution, not a substitute for submitting things upstream..
<superm1> This original purpose has worked very well for Dell and the different distributions shipped on pre-installed laptops (RHEL, SuSE, & Ubuntu).  Fortunately, the way the DKMS framework was assembled, it has gained several other very useful purposes.
<superm1> Starting with Intrepid, all users of proprietary graphics kernel modules will be using DKMS to maintain the kernel module that ships with the driver.  This means it will be significantly easier to add newer drivers to Ubuntu releases and less of a maintenance overhead for the kernel team that was previously maintaining linux-restricted-modules and it's 150MB+ source package.
<superm1> Also, it ends up being a useful debug tool.  DKMS allows the creation of packages with both the source and binary modules self contained.  This means that users will be able to install a package and then be able to test it on a variety of kernels other than the original intended kernel.
<superm1> So, now that you have a pretty basic idea of what DKMS has been used for thus far, I'll give you an example of how to use it.
<superm1> The first example we're going to go into is the basic updated ethernet driver.  As you may have heard, Dell is going to be shipping Ubuntu on it's Vostro line of laptops.
<superm1> There unfortunately was a very bad bug that was encountered shortly after 8.04 was released preventing the ethernet adapter from working.  A patch was developed that is now included in 8.04.1.  Users will be able to upgrade to 8.04.1 after receiving their machines, but performing the upgrade without an ethernet driver is a little bit difficult.  Here's an overview of how we got around that issue.
<superm1> Ideally i'd like people to follow along
<superm1> i'll try to give the steps as generically as possible so that you can try this on the last few ubuntu releases
<superm1> 1) Install DKMS on your machine.  The version in hardy is a little bit old, but the version in Intrepid is up to date.  Until the backport is ready, I'd recommend grabbing the latest version from http://linux.dell.com/dkms/.  This is the same package that is in Intrepid.
<superm1> 2) Grab the realtek ethernet driver that shipped with 8.04 from http://kernel.ubuntu.com/git?p=ubuntu/ubuntu-hardy.git;a=blob_plain;f=drivers/net/r8169.c;hb=46d798f10f53bf6814cb6b429f45e660b0a9aee4 .  Save this file as r8169.c
<superm1> 3) Grab the patch(es) that needed to be applied to make the card function: http://pastebin.com/m5fed36a7 .  Save this as r8169-8.04.1.patch
<superm1> 4) Create two directories, /usr/src/r8169-2.2LK/ and /usr/src/r8169-2.2LK/patches.  I'll explain a little bit later why this naming scheme was chosen.
<superm1> 5) Copy r8169.c into /usr/src/r8169-2.2LK
<superm1> 6) Copy r8169-8.04.1.patch into /usr/src/r8169-2.2LK/patches
<superm1> 7) Create a basic kernel Makefile in /usr/src/r8169-2.2LK/Makefile.  You can grab the one I've made from http://pastebin.com/f2ed728e2 . This Makefile just lists the different objects that get linked together.  Some modules won't have very complex Makefile's (such as this one)
<superm1> 6) Now, we'll craft a DKMS control file to explain what to do with these pieces.  This file explains to DKMS exactly what we will be doing, what the packages is called, what the modules are named etc.  Take a look at http://pastebin.com/f5bea78be
<superm1> so, first the obvious ones:
<superm1>  a) The PACKAGE_NAME field describes the name of the DKMS package we are making
<superm1>  b) The PACKAGE_VERSION field describes the version of the package we are making.
<superm1>  c) the CLEAN line explains exactly what needs to be cleaned up and how to do it
<superm1> you'll find these 3 in all packages dkms control files
<superm1>  d) BUILT_MODULE_NAME[0] is for the first kernel module we are creating. 
<superm1> this is listed as a vector because you can build many modules from a single dkms packages
<superm1> *package
<superm1> e) PATCH[0] lists the patches that get applied to the pristine module prior to the build
<superm1> f) DEST_MODULE_LOCATION[0] describes where we will be installing the module to on the system.  By default updates/ is higher priority than the normal kernel driver locations when module dependencies are calculated
<superm1>  g) AUTOINSTALL describes whether we will automatically rebuild and reinstall on newer kernel releases.  I'll talk more about this later.
<superm1> There are a lot of other very complicated things you can do with DKMS, so this is a very basic example only listing the above options.
<superm1> so now that you have an idea of what's in the control file
<superm1> 7) Place this DKMS control file in /usr/src/r8169-2.2LK/dkms.conf
<superm1> well I suppose i should have asked for questions sooner, so i'll interrupt this and answer quickly before carrying on:
<superm1> QUESTION: what are we doing precisely ?
<superm1> so what this is doing is replaying how a DKMS package was created a few months ago in a real life situation
<superm1> this example actually happened, so this is how we (Dell) got around it
<superm1> okay i'll pick back up then - feel free to interject as necessary as i keep going
<superm1> 8) Add the package to the DKMS build system.  All the pieces are ready now, so this registers the package on  the system.
<superm1> sudo dkms -m r8169 -v 2.2LK add 
<superm1> 9) Install the headers for your current running kernel so that we can build against it.
<superm1> sudo apt-get install linux-headers-`uname -r`
<superm1> 9) Build the module.  Once you've added a module, bash-completion will allow you to tab complete a lot of these fields.
<superm1> sudo dkms -m r8169 -v 2.2LK build
<superm1> the build process should complete without any error
<superm1> this means that the kernel modules are "available" on your system, but not in use anywhere
<superm1> 10) Now our module is built and ready to use.  We can build for additional kernel versions by specifying the -k field.  If we were looking for some widespread testing of this module, we may want to put it on our PPA.  The latest version of DKMS has support to build debian source packages just for this purpose.  
<superm1> sudo dkms -m r8169 -v 2.2LK mkdsc
<superm1> those familiar with packaging (or at least have attended recent talks), should know that this is the file that describes your package
<superm1> 11) You will have a dsc and tar.gz.  You will just need to create a .changes file that you can dput (outside the scope of this howto) and sign it with your GPG key.
<superm1> 12) If you wanted to distribute this without a PPA, you could issue this command to build a binary deb for users to use:
<superm1> sudo dkms -m r8169 -v 2.2LK mkdeb
<superm1> similar functionality is available for creating driver disks, and packages for other distributions.  of course this is an ubuntu centric talk, so I'll only talk about mkdeb/mkdsc
<superm1> Either way, you will have files in /var/lib/dkms/r8169/2.2LK that can be distributed.  Now the thing that I didn't talk about here was that AUTOINSTALL directive.
<superm1> If you know that this is only necessary for a single kernel release, you may want it to not AUTOINSTALL.
<superm1> If you know exactly when the support is available, there is functionality to indicate when the package is marked OBSOLETE_BY.  This and other advanced features are discussed a bit more in detail on the man page.
<superm1> For our case we knew that the support was available in the next kernel ABI, so we were able to define OBSOLETE_BY to be 2.6.24-18.  This means that the DKMS package will remain on the system, just dormant.  Users will be able to remove it themselves or let it sit around without doing any harm
<superm1> Now, For some other examples of how DKMS is used, I'd recommend taking a look at the source for fglrx-installer, nvidia-drivers-*, lirc, and qemu.
<superm1> the intrepid versions of each of these packages uses DKMS to rebuild kernel modules independently of the running kernel
<superm1> hey each adapt DKMS into their packaging quite well.
<superm1> *they
<superm1> there is also a video and a thorough wiki page describing some more about DKMS usage at https://help.ubuntu.com/community/Kernel/DkmsDriverPackage and http://blog.phunnypharm.org/2008/07/dkms-presentation.html respectively.
<superm1> That's about what I wanted to walk through.  The biggest take away is that by using DKMS, the hardest part of delivering fixed modules should be coming up with the fixes, not necessarily figuring out the right way to package them and keep them maintained
<superm1> so, does anyone have any questions?
<superm1> well then i suppose i'll end up wrapping up a bit early :)  Feel free to send any questions that you may come up with later to the DKMS mailing list or to poke me on IRC if it's something small/quick
<mok0> thanks, superm1
<superm1> well since there was one more question, "<tacone> giving a little overview on what dmks precisely is"
<superm1> so i'll try to go a little more in detail
<superm1> so it's a Dell developed framework for situations just like these.  Before we were dealing with Ubuntu, getting some very large patches into RHEL kernels w/ stable ABIs was very difficult
<superm1> often to enable things like audio, you would have very invasive patches that would be rolled into the next RHEL release, but the schedule for releases fit well outside the schedule for launching the workstation or laptop
<superm1> working closely with Ubuntu, this is a lot easier and not necessary for those particular purposes.  Ubuntu does have a stable release update policy, so rather than shipping a lot of DKMS'ified drivers, it's just when the corner cases come about that we really use it
<superm1> in trying to be good netizens, this is one of the things that we try to contribute outside of simple hardware enablement
<superm1> <tacone> Q: what's the point of DKMS ? to compile a new driver against a new kernel release automatically ?
<superm1> yeah, that's the biggest advantage that can get taken away with it
<superm1> since it's an extendable framework, you can even include patches in your dkms package so that newer kernels that would normally prevent compiling due to ABI changes don't fail.
<superm1> alberto added a patch to the nvidia-glx 177 package for this purpose on the 2.6.27 kernel
<superm1> also to my knowledge, Sun has started to use it for virtualbox modules
<superm1> Q: <tacone> you can prevent a compile from failing ? <tacone> vodoo ?
<superm1> yes, and yes.  well actually no.  but if you know that this will be used with other kernels, and know where it will be breaking, you can include patches directly in the DKMS patch for those different kernels.  This means that lets say RHEL had 2.6.18 and we had 2.6.24.  If you wanted to use this same package on both systems, you could create a patch that allows it to compile against 2.6.18 that only gets applied when the user tries to u
<superm1> se it on 2.6.18
<superm1> <albert23> QUESTION: can I use DKMS to build an i386 deb on amd64?
<superm1> Hum.  That's an interesting question.  I can't say I've experimented personally on doing this.  I would think so, but you will have to modify your build line and/or Makefile to be sure to choose a 32 bit compiler
<superm1> i'd be more confident of it finishing properly if you were to do it in an i386 chroot
<superm1> Q: <tacone> so DKMS allows you to distribute patches for different kernels into just 1 package, right ?
<superm1> Yes.  Of course if there is a newer kernel that was released needing a new patch, you can't predict the future in the package.  You'll have to release a new package that includes that new patch.  You don't have to worry about breaking earlier kernels though since it's conditionally applied
<superm1> this is why there are systems like the PPA system available though. you'll just need to publish a newer source package with that patch included
<superm1> in the cases of the more widespread use (like  fglrx and nvidia), such patches would be done in formal stable release updates
<superm1> OK, any more questions?