Ubuntu Wiki

   1 [20:01] <jamespage> Hi Everyone o/
   2 [20:02] <jamespage> Welcome to the Java library packaging with maven-debian-helper Ubuntu Developer Week session
   3 [20:02] <jamespage> So - organisational bits first:
   4 [20:02] <jamespage> Please ask questions whenever you like on #ubuntu-classroom-chat and please prefix your question with 'QUESTION:'  I'll try to keep and eye out
   5 [20:02] <jamespage> All session logs will be uploaded and accessible from https://wiki.ubuntu.com/UbuntuDeveloperWeek - so if you need a reminder of something covered in the session then please look there.
   6 [20:03] <jamespage> My name is James Page and I've been contributing to Ubuntu since September last year
   7 [20:03] <jamespage> I work for Canonical and I'm a member of the Ubuntu Server team so apologies if some of this session is a little server centric :-)
   8 [20:04] <jamespage> This session is intended to give you an overview of packaging Java libraries for Ubuntu - specifically Java projects that use the Maven build system.
   9 [20:04] <jamespage> So first things first - why package Java libraries for Ubuntu?
  10 [20:05] <ClassBot> Daviey asked: Java tradionally has a bad reputation on Linux, why is this?
  11 [20:05] <jamespage> Timely question - lets cover that one first
  12 [20:05] <jamespage> I think it due to the way the Java projects are typically developed
  13 [20:06] <jamespage> Because the Java Virtual Machine abstracts the developer away from the underlying operating system
  14 [20:06] <jamespage> they are free to load that virtual machine with exactly the code they want
  15 [20:07] <jamespage> dependent libraries are often bundled in a variety of nasty ways
  16 [20:07] <jamespage> which means that its hard to pull out common libraries and base line in the way a Linux distro likes todo.
  17 [20:08] <jamespage> not impossible - just hard
  18 [20:08] <jamespage> to a certain extent the toolsets exacerbate this behaviour
  19 [20:09] <jamespage> so back to my original question - why package Java libraries for Ubuntu?
  20 [20:09] <jamespage> Well from my perspective its about delivering Java applications onto the Ubuntu platform;
  21 [20:09] <jamespage> this would include things like Tomcat, Jenkins, Hadoop, Eclipse etc...
  22 [20:09] <jamespage> (some of which we have in Ubuntu, some of which we don't - yet)
  23 [20:10] <jamespage> However these applications need to rely on the Java libraries that are packaged for Ubuntu
  24 [20:10] <jamespage> in the same way that other libraries are packaged once and then used by all applications that need them we aim todo the same with Java libraries.
  25 [20:11] <jamespage> So having a broad, well maintained set of libraries is key to both maintaining the existing Java applications and delivering new applications into Ubuntu.
  26 [20:11] <jamespage> This does not always align well with the way that most Java project work;
  27 [20:12] <jamespage> as I just discussed each project can pretty much load the Java Virtual Machine with the code then want
  28 [20:12] <jamespage> which means they don't have to give consideration to other Java projects - because they will be isolated in the same way
  29 [20:13] <jamespage> however by maintaining a single set of libraries we try to bring some of the Linux distro goodness to the Java world
  30 [20:13] <jamespage> So - next bit is a little dull but important
  31 [20:13] <jamespage> Java libraries should follow the Debian Java Policy - see http://www.debian.org/doc/packaging-manuals/java-policy/ for the full details.
  32 [20:14] <jamespage> In fact its probably worth pointing out now that 90%+ of the Java libraries come unchanged for Debian
  33 [20:14] <jamespage> As a quick 101:
  34 [20:14] <jamespage> Library packages should be named libXXX-java - for example libcommons-io-java
  35 [20:14] <jamespage> Documentation should be in a separate package libXXX-java-doc
  36 [20:14] <jamespage> Libraries (jars in the case of Java) should be installed into /usr/share/java - publishing Jar's into a single location aids with discover-ability etc...
  37 [20:14] <jamespage> and where possible into /usr/share/maven-repo - more on this in a bit.
  38 [20:15] <jamespage> Fortunately the toolset around generating Java library packaging for Maven projects helps out quite alot with this so you should find it fairly easy to generate a policy compliant library.
  39 [20:16] <jamespage> Just in case you are not familiar with Maven its probably the most popular software project management tool used by Java projects
  40 [20:16] <jamespage> Its much more than just a build tool - hence 'Software Project Management'
  41 [20:17] <jamespage> through a pretty extensive range of plugins it integrates with most SCM's and issue trackers allowing easy automation of defect tracking, release processes, publishing of project artefacts etc...
  42 [20:18] <jamespage> however in Ubuntu it does get pretty much relegated to being a build tool - most of value add is not applicable and is mainly used by the upstream projects
  43 [20:18] <jamespage> It operates by convention; if you stick your code in the right place Maven will find it and build it; it will also have a look for test code and compile and run that as well.
  44 [20:19] <jamespage> Its uses project object model files - POM's
  45 [20:19] <jamespage> to define various metadata about the software project including its namespave, name, dependencies, build process, author, licensing ....
  46 [20:20] <jamespage> the list goes on and is incredibly rich - this information is really useful when packaging
  47 [20:20] <jamespage> but depth is not always great :-(
  48 [20:20] <jamespage> So I'm now going to give a quick demo of packaging a basic Java library using maven-debian-helper
  49 [20:20] <jamespage> To follow the demo you will need to log into the following server
  50 [20:20] <jamespage> ssh guest@ec2-46-137-134-25.eu-west-1.compute.amazonaws.com
  51 [20:20] <jamespage> password should be guest
  52 [20:21] <jamespage> I'm going to be using a tool called byobu so that we can all see the same session.
  53 [20:21] <jamespage> So as it takes a bit of time I've already setup the packaging environment on this Ubuntu server;
  54 [20:22] <jamespage> If you needed to do it
  55 [20:22] <jamespage> sudo apt-get install packaging-dev default-jdk maven-debian-helper javahelper apt-file aptitude
  56 [20:22] <jamespage> packaging-dev (thanks bdrung) is a new package which is intended to setup all the basics for package development;
  57 [20:22] <jamespage> we also need the Java Dev Kit and the java specific helpers (plus some other tools)
  58 [20:22] <jamespage> So next step - generate some basic packaging using mh_make;
  59 [20:23] <jamespage> we are going to pull a upstream tarball from github and then do some basic packaging
  60 [20:24] <jamespage> metainf-services is a very simple library that helps generated metadata for jar files during packaging
  61 [20:24] <jamespage> so as you can see we just have a pom.xml file and a src dir
  62 [20:25] <jamespage> next we use mh_make
  63 [20:25] <jamespage> mh_make tries to guess most things but it does give you the option to change stuff
  64 [20:26] <jamespage> so we are going to run tests and generate API docs
  65 [20:26] <jamespage> mh_make uses apt-file to search for any missing dependencies
  66 [20:26] <jamespage> if it can't find something it needs in /usr/share/maven-repo it will search using apt-file and try to make a recommendation
  67 [20:27] <jamespage> So this bit is quite important
  68 [20:27] <jamespage> when the package is built it gets deployed into /usr/share/maven-repo twice
  69 [20:27] <jamespage> once under the original version - 1.2 in this case
  70 [20:28] <jamespage> and once under a fixed label - this is normally debian or 1.x/2.x if multiple versions of a library are packaged
  71 [20:28] <jamespage> this means that other libraries can 'fix' on a version which can then be changed under the hood if a new version of the library is released
  72 [20:28] <jamespage> without having to update all pom files
  73 [20:28] <jamespage> more on this in a bit
  74 [20:29] <jamespage> mh_make also makes a guess as to with plugins are not useful for packaging
  75 [20:29] <jamespage> this last one is unknow - infact its used for publishing a micro-site to github - which we don't need either
  76 [20:30] <jamespage> mh_make then generates the base packaging
  77 [20:30] <jamespage> you will notice that it used licensecheck to search for useful information on copyright and licensing - we'll see the results of that in a mo
  78 [20:31] <jamespage> So I'm going to make this into a bazaar branch to help out a bit
  79 [20:31] <jamespage> So lets take a look around:
  80 [20:32] <jamespage> So using the information that licensecheck found in the headers mh_make has had a stab at generating a copyright file
  81 [20:32] <jamespage> its not bad - normally this needs a few tweaks to get it exactly right but it does most of the hard work
  82 [20:32] <jamespage> The maven.*rules files are used by maven-debian-helper to transform the Maven pom.xml files during the build of the project;
  83 [20:33] <jamespage> so in maven.ignoreRules you can see the two plugins that we told mh_make to ignore - these get transformed out of the pom.xml during the build
  84 [20:34] <jamespage> maven.rules is pretty simple in this case - this will create a 1.x version alongside the 1.2 version in /usr/share/maven-repo
  85 [20:34] <jamespage> so we also get a maven.cleanIgnoreRules
  86 [20:34] <jamespage> this is used when the clean target is called for the project
  87 [20:35] <jamespage> typically this requires alot less dependencies so it may be a longer list of exclusions
  88 [20:35] <jamespage> for this package its OK for it to be the same as maven.ignoreRules
  89 [20:35] <jamespage> so I'll just do a quick change to the changelog and we are good to go
  90 [20:36] <jamespage> You will notice that mh_make has used  Java Demo <java.demo@ubuntu.com>
  91 [20:36] <jamespage> this gets picked up from DEBEMAIL and DEBFULLNAME environment variabled - I set these up earlier
  92 [20:37] <jamespage> so source package build first
  93 [20:38] <jamespage> maven-debian-helper patches/unpatches the pom.xml file during the process
  94 [20:38] <jamespage> the default target for building the package is 'package' - however for more complex packages 'install' may be more appropriate.
  95 [20:38] <jamespage> generating javadoc
  96 [20:38] <jamespage> (twice - this is a bug!)
  97 [20:39] <jamespage> automatically determining dependencies for the binary packages
  98 [20:40] <jamespage> and done - a few lintian warnings but nothing unsolvable
  99 [20:40] <jamespage> so we now have a built package \o/
 100 [20:41] <jamespage> as we discussed earlier the package deploys 1.2 and 1.x artefacts to /usr/share/maven-repo
 101 [20:42] <jamespage> it also deploys to /usr/share/java - this is to support ant/javahelper based build and applications that require this library
 102 [20:42] <jamespage> and also a -doc package containing the generated API
 103 [20:42] <jamespage>  just to prove that we have created something that at least installs
 104 [20:43] <jamespage> Obviously this is a relatively simple example;
 105 [20:43] <jamespage> however the concepts covered in this session can be used to build up complex dependency chains of packages to support large Java applications like Jenkins.
 106 [20:43] <jamespage> Maven actually makes packaging for Ubuntu easier;
 107 [20:44] <jamespage> because it has a standard way of expressing dependencies and a rich metadata model it is easier to automate the production of packaging
 108 [20:44] <jamespage> packages that use ant are more challenging as they express dependencies in a million bespoke ways
 109 [20:44] <jamespage> (unless they are using a dependency management system like ivy which can make things a little easier).
 110 [20:45] <jamespage> Questions?
 111 [20:46] <ClassBot> coalitians asked: so mavens central repository is never used ?
 112 [20:46] <jamespage> good question - the local Debian repo in /usr/share/maven-repo is used and Maven is executed fully offline
 113 [20:47] <jamespage> maven-debian-helper handles this as part of the packaging build rules
 114 [20:47] <jamespage> so if you switch back to the demo you can see that the debian/rules file is very simple - all of the logic in encapsulated in cdbs classes
 115 [20:48] <jamespage> Any more for any more?
 116 [20:48] <jamespage> OK
 117 [20:48] <jamespage> I hope that you have found this session useful;
 118 [20:49] <jamespage> I normally hang out on #ubuntu-server and #ubuntu-java so if you have any more questions please feel free to ping me.
 119 [20:49] <jamespage> Thankyou and goodnight!