Launchpad Entry: server-karmic-eucalyptus-in-main
Created: June 2nd, 2009
Contributors: ThierryCarrez
Packages affected: lots
Summary
Eucalyptus, as part of the Ubuntu Enterprise Cloud, needs to move to main to be properly supported by Canonical. That means that all runtime and build-time dependencies (and all dependencies of the dependencies) must also move to main. This specification tracks which packages will need to move to main and efforts made to reduce the total number as much as possible.
Release Note
Eucalyptus is now part of the Ubuntu "main" repository, and fully supported by Canonical.
Rationale
Eucalyptus is currently in universe, together with most of its Java library dependencies. Since Eucalyptus is part of the Ubuntu Enterprise Cloud product, it needs to be properly supported by Canonical, which means move to the "main" repository. However, packages in "main" also need to have all their runtime and build-time dependencies in "main" as well. Java software in general heavily makes use of other Java libraries, so this quickly snowballs into a large number of packages to move to main. Furthermore, there isn't so much Java-based software in main already (OpenOffice.org, tomcat6) so even some basic Java libraries need to be moved.
User stories
- Dave has been trying the UEC technology preview in 9.04 and aims to deploy UEC with the 9.10 release. As it will no longer be a technology preview, it needs to be properly supported.
Assumptions
This spec does not take care of any other non-Java MIR that Eucalyptus may need. iscsitarget for example could be a new 1.6 dependency, and it is in universe currently.
Design
We need to try to keep the number of packages as small as possible in order to avoid MIR team overload. Some hacks (like duplicating some greedy source packages to avoid bringing lots of unnecessary build dependencies) are worth the trouble if they save a significant amount of packages. However Debian doesn't care about keeping the number of dependency packages small, so those hacks don't really make sense from a Debian perspective.
Avoid packages difficult to maintain
- Large software stacks
- JBoss (pulled through Hibernate)
- Glassfish (pulled through glassfish-javaee and glassfish-toplink-essentials, used as build-deps)
Proposed actions for package number reduction
- Use a specific eucalyptus-javadeps with hibernate/ehcache/spring: saves 60+ packages
Remove dependency from libclassworlds-java -> maven-ant-helper: saves 8+ packages
- Avoid dependencies on Glassfish packages, use geronimo-specs-* instead
Resulting list (without those from RefactorEucalyptusJavadepsSpec)
package* denotes a new package in karmic
Direct dependencies
antlr3 eucalyptus-commons-ext-java* (eucalyptus-commons-ext-java) groovy janino asm2 (libasm2-java) libaxiom-java* (libaxiom-java) c3p0 (libc3p0-java) cglib2.1 (libcglib2.1-java) libcommons-cli-java libcommons-fileupload-java commons-io (libcommons-io-java) libcommons-jxpath-java dnsjava* (libdnsjava-java) drools* (libdrools-core-java) excalibur-logkit (libexcalibur-logkit-java) libezmorph-java geronimo-ejb-3.0-spec* (libgeronimo-ejb-3.0-spec-java) geronimo-j2ee-connector-1.5-spec* (libgeronimo-j2ee-connector-1.5-spec-java) geronimo-jms-1.1-spec (libgeronimo-jms-1.1-spec-java) geronimo-jpa-3.0-spec* (libgeronimo-jpa-3.0-spec-java) geronimo-jta-1.0.1b-spec (libgeronimo-jta-1.0.1b-spec-java) geronimo-jacc-1.1-spec* (libgeronimo-jacc-1.1-spec-java) geronimo-interceptor-3.0-spec* (libgeronimo-interceptor-3.0-spec-java) libgoogle-collections-java gwt* (libgwt-java) javassist (libjavassist-java) jetty* (libjetty-java) libjibx-java libjson-java jug* (libjug-asl-java) mvel* (libmvel-java) netty* (libnetty-java) libslf4j-java wss4j* (libwss4j-java) libxml-security-java
1st-level Dependencies of deps
ivy [for groovy] jruby1.1 [for eucalyptus-commons-ext-java] junit4 [for groovy] libaopalliance-java [for eucalyptus-commons-ext-java] asm (libasm-java) [for libcglib2.1-java, eucalyptus-commons-ext-java] aspectwerkz2 (libaspectwerkz2-java) [for libcglib2.1-java] libcommons-attributes-java [for eucalyptus-commons-ext-java] libjamon-java [for eucalyptus-commons-ext-java] jexcelapi (libjexcelapi-java) [for drools] mockobjects (libmockobjects-java) [for groovy] ow-util-ant-tasks (libow-util-ant-tasks-java) [for libasm2-java] qdox (libqdox-java) [for libjibx-java] stringtemplate (libstringtemplate-java) [for antlr3] swt-gtk (libswt-gtk-3.4-java) [for gwt] libxstream-java [for drools, groovy, eucalyptus-commons-ext-java]
2nd and more level
commons-vfs (libcommons-vfs-java) [for ivy] concurrent-dfsg (libconcurrent-java) [for libaspectwerkz2-java] libhamcrest-java [for junit4] jarjar (libjarjar-java) [for libaspectwerkz2-java] jmock (libjmock-java) [for libqdox-java] libjoda-time-java [for libxstream-java] jrexx (libjrexx-java) [for libaspectwerkz2-java] trove (libtrove-java) [for libaspectwerkz2-java] asm3 (libasm3-java) [for libjarjar-java] easymock (libeasymock-java) [for libhamcrest-java]
Total
Total: 60 packages to MIR
Implementation
Package eucalyptus-commons-ext.jar
eucalyptus-commons-ext.jar is a Eucalyptus upstream JAr file containing the subset of hibernate needed for Eucalyptus (without all the JBoss hooks the libhibernate3-java package contains). Same for ehcache and Spring.
Implement other hacks
All the other number-reducing hacks should be implemented:
Remove dependency from libclassworlds-java -> maven-ant-helper: DONE
- Remove jmock and libconcurrent dependency on kaffe: DONE
Potentially problematic packages
Here are exceptions to the common "simple Debian JAR package" case:
Not just Java libraries
- janino (binary)
- groovy (binary)
- swt-gtk (-jni native libraries)
Security concerns
- gwt
- jetty6
Not updated to default-java stuff yet
- commons-vfs
- libjoda-time-java
- jarjar
- asm3
- aspectwerkz2
- jrexx
- trove
Not in Debian yet
- libaxiom-java
- dnsjava
- netty
- jug
- mvel
- wss4j
- geronimo-*-spec
- jetty6 (work in progress)
- gwt
Unconfirmed sets
commons-cli commons-fileupload commons-io commons-jxpath excalibur-logkit commons-attributes ezmorph google-collections json antlr3 / stringtemplate drools / jexcelapi euca-commons-ext / jruby1.1 aopalliance jamon
Expected exceptional ones in there:
- euca-commons-ext: NOTINDEBIAN, DUPLICATION, SECURITY (Spring)
- jruby1.1 : BINARY
Process MIR sets
The EucalyptusInMainSpec/Packages page lists all the source packages that need to be processed. During a meeting with MIR team members we confirmed that rather than filing individual MIRs, any particular package should get tagged in the list for more in-depth review. A source package with no tags is a regular Java library without any specific particularity.
Test/Demo Plan
No Eucalyptus-Java-dependencies-related components mismatch should appear when the Eucalyptus update is pushed to main.
Unresolved issues
None.
BoF agenda and discussion
UDS discussion
Current status
- 99 packages which would need to move to main.
Java in main is currently driven by OpenOffice. Tomcat6 is there, but has few dependencies.
Future
- Eucalyptus version for Karmic should have fewer dependencies anyway from upstream refactoring.
How can the number of build/runtime dependencies be reduced?
- Get rid of anything not necessary
- Geronimo spec packages (already provided by glassfish-j2ee)
- Move parts into Eucalyptus code
- Refactor packages (e.g. hibernate pulls in a lot of stuff)
- Requires multiple source packages or only to promote some resulting binaries to main.
- (doko recommends duplicating the source package and documenting it)
- Requires multiple source packages or only to promote some resulting binaries to main.
What could be expected to be refused inclusion to main
- Active upstream criteria
- Annogen
- OpenSAML
- WSS4J?
- Known to be insecure
- Nothing obvious
- Partial builds (because of non-free dependencies)
- Mule
- We don't really care
Size issues (adding to default CD)
- GWT is 6MB of binary
- 1.6 should be more granular and work with existing java packages more
- JRuby is 3MB
- Just be a build-time dependency of GWT
- libjgroups-java, 3MB. Not needed ?
- glassfish-toplink-essentials, 2.4MB, comes from hibernate
- Avoiding JDK would save about 50MB of CD space
- Avoiding GCJ would save about 20MB
Can we use the new Spring packaging from Debian?
- Not included yet in Debian, but work is underway
- Might make too hard choices for us ? Testing from Eucalyptus side needed.
Safety net
- Keep a eucalyptus-javadeps package to drop in any problematic parts which can't be fixed (hibernate?).
- One tiny part of Drools
Action Summary
- Which Eucalyptus version we'll use in Karmic and its dependencies?
- Can we use Spring from Debian?
- What can we do to reduce Hibernate size?
- Will it be acceptable to move JBoss to main?
- JDK dependency investigation
- Ask Colin about blacklisting for the server seed only to solve GCJ issue
- Once it's known what is required, start preparing/filing MIRs to avoid a last minute panic.