Ubuntu Developer Summit Proceedings

In October 2010 in Orlando, Florida, many Ubuntu contributors, upstreams and interested organizations joined us at the Ubuntu Developer Summit to plan the new release, Ubuntu 11.04 Natty Narwhal.

This page summarizes many of the outcomes of the event, and for each track there is a link to further detailed notes. Please note: these are proceedings and plans, and some of these things may not get completed as planned for whatever reason. As such, please read this list as a set of goals, and not a promise of what Ubuntu 11.04 will include.

Thanks to everyone who participated in the Ubuntu Developer Summit, and lets make this a rocking release!

Ubuntu The Project

• The Upstream Contacts project will perform six-monthly surveys to get a finer impression of upstream perspectives on Ubuntu. There will be a continued focus on every project having an Upstream Contact assigned.
• There was a discussion about the components needed to allow community developers to quickly build new web applications for Ubuntu projects. These come from the common aspects of loco-directory, summit, harvest and awstrial. A project has been formed to build a Django based solution using Bazaar and Launchpad to help community teams provide these resources.
• Developer advocacy will continue to be a focus in Natty with work going into creating a demo script for explaining how development works, creating screencasts and reviewing our existing documentation.
• Accessibility was a key focus at this UDS and in support of the plan to build accessibility support into Unity for 11.04, the community Accessibility Team reviewed and improved their on-boarding documentation, and will be providing bug triage and testing resources throughout the Natty cycle. In addition to this, accessibility tests will be appended to the regular release tests and ISO testing.
• The sponsorship queue in this cycle will get some significant support with a new "Patch Pilot" scheme. The scheme will work by providing Canonical engineering staff throughout the month who will focus on reviewing new contributions made by developers who are working towards being approved Ubuntu developers. This should provide a faster review time and on-ramping of Ubuntu developers.
• Work will continue in the Translations community and as part of this work an introductory video will be produced to explain how people can get involved in the community, a language packs schedule has been defined, there will be a series of translations training sessions throughout the cycle, and a first iteration of translations.ubuntu.com will be produced.

• The community made plans to produce a Global Directory, inspired by the LoCo directory, which shows all teams in the Ubuntu community. As part of this work a meeting tracker which integrates with fridge.ubuntu.com has been planned.

• The Release Team met and agreed to update Process pages to include explicit post mortem discussion in future, clarify roles in the release pages, merge release schedules into a master table, and review the Natty cycle and make changes where appropriate.

• The Leadership Code Of Conduct was agreed to play a formal role in leadership in Ubuntu, and the translations community will be porting it to DocBook so it can be translated by the community.

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/UbuntuTheProject

Application Developers

• The Ubuntu One team will be delivering an application developer API that application authors can use to build Ubuntu One support into their apps. Core focus for this API will be desktop + web + mobile.
• Natty will see significant work in the area of gestures. A high-level gesture language will continue to be developed, with support in GTK/Qt apps. The GEIS 2.0 gesture language resulted from experience with GEIS 1.0, and it addresses awkward aspects of usage so far.
• The Ubuntu Developers' Manual team will organize a sprint or other focused period of time to produce the manual, perhaps hiring a technical writer or editor as a consultant.
• The decision was taken to deliver apps form extras.ubuntu.com into /opt, and under an new namespace in order to avoid present and future naming collisions. This will require work on the Python build system to support byte compiling in /opt, as well as to solve other issues.
• Long term support for full application sandboxing was also discussed. It was determined that sandboxing will be a multi-release effort, and will not appear in Natty.
• The Quickly ubuntu-application template will be enhanced to make the user edited code cleaner, and to make it easier to access widgets and signals from code.

• Game development was discussed, but will not receive any significant focus outside the existing Quickly PyGame template. A lack of artwork was identified as a blocking issue, but no one committed to creating a repository of free game art.

• The Quickly Widgets project will not receive any significant focus, and will remain a community driven project.

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/ApplicationDevelopers

Package Selection and System Defaults

• Unity will provide the default desktop experience for Ubuntu Desktop

• Gnome 3.0, including Gnome Shell, will be included in this release.

• Software Center will have two major feature updates. These updates will allow Ubuntu users to:

  1. donate money directly to providers of free software in the Ubuntu Software Center
  2. access and provide ratings, plus reviews, for all software in the Ubuntu Software Center

• In accordance with the Ubuntu Promise, Unity will be made accessible with a focus on the launcher, the panel, and the indicators that go with it.

• We will target Banshee as the default music player.

• The boot experience for Ubuntu Server will be improved through better integration with upstart and plymouth.

• In our on-going effort to support community-driven open source server related projects, the Ubuntu Server team will ensure the integration of Drizzle, a "Lightweight SQL Database for Cloud and Web".

• There will be further focus on automation of server and cloud image testing as a result of successful automation of ISO testing during the Maverick cycle using Hudson, which will also be packaged in this cycle.

• The Ubuntu Server team will also package Tomcat 7 and work with OW2/JOnAS to evaluate packaging of a full Java Enterprise Edition (EE) stack.

• In Ubuntu 11.04, we will re-assess our networking stack from the kernel up to the user interface layer, with a singular goal of giving the best network experience to Ubuntu users to date.
• Natty development will open with gcc-4.5 as the default toolchain, with an aggressive goal of shipping gcc-4.6 by release.

• Linaro will create a new ARM-targeted "developer" image. This image will be a strict superset of the currently produced headless image, to reduce maintenance, and will include developer tools and other useful packages generally interesting/useful to ARM developers.

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/PackageSelectionAndSystemDefaults

Cloud Infrastructure

• For Ubuntu 11.04, we will continue to support and integrate key cloud infrastructure features delivered by Eucalyptus.

• We will look at providing an installation service for physical cloud node deployments that will automatically install, configure, and integrate the new node into an existing Ubuntu-based cloud infrastructure.

• We will make numerous improvements to the Ubuntu Cloud images, that include expanding support for use on Cluster Compute Images on Amazon EC2 and added support for Rightscale in cloudinit.

• In Natty, we will tighten our integration with Puppet by:

  • allowing a new system, installed by the Ubuntu's cloud installation service, to automatically register itself with the puppet service, so that it can be configured as the expected cloud component.
  • packaging mcollective to get a scalable distributed command infrastructure (e.g. trigger puppet runs).
  • writing puppet modules to configure Ubuntu-based cloud components

• The Ubuntu Server team will improve the integration of both Openstack and Hadoop, as well as package Open vSwitch and zeromq.

• In Ubuntu Server 11.04, we will look to provide support for Linux Containers in Ubuntu-based cloud deployments by helping to make LXC ready for production.

• Our cloud image store is one of the key differentiators that Ubuntu offers. During the Natty cycle, we will attempt to meet the needs of our users, with a focus on Enterprise deployments, ISVs, and ISPs.

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/CloudInfrastructure

Multimedia

• Linaro Multimedia Team investigate options to optimize jpeg decoding transparently on ARM systems that have a NEON unit and hardware support
• The Online Services team will work on integrating ubuntu one with shotwell
• Linaro Graphics Team will work on optimizing runtime behaviour on ARM for vector graphic libraries like cairo, skia, but also toolkits like qt by automatically selecting the best backend and using NEON units were possible
• The Linaro User Platforms team will provide convenience images to support graphics and multimedia development efforts within and outside of linaro.

• The Linaro Graphics Team will continue its work on improving availability direct rendering support for graphics on the various ARM SoCs

• The Touch Team will work explore various creative ways to use touch interfaces in ubuntu; among others, this includes connecting the ipad as making a touch app to control your TV from ubuntu
• Work to improve low latency audio will be conducted to make jack work with pulseaudio.
• The Linaro Graphics Team will continue to extend the graphics benchmark software and tools available in ubuntu and linaro; among other approaches, trace support along the line of cairo tracke for more toolkits than cairo is being investigated.
• Depending state of touch screen support the desktop team will stay on xorg 1.9 or move to 1.10
• The Desktop Team will work on strengthening the use of gallium by default; which hardware will make use of gallium drivers by default is subject to upstream and user feedback
• The Desktop Team will ensure that xvfb becomes more solid as this is a requirement of the security team
• The Design Team will continue to investigate how color management in ubuntu can be streamlined
• Ubuntu will work on improved multi monitor support, especially for unity
• The Linaro Multimedia Team will work on improving support for ARMs low-power mode when using pulseaudio; this will involve investigating adding smart alsa-sink buffer tuning while still ensuring that real time events will not get delayed
• The Ubuntu ARM team will make GLES 2.0 the default GL standard used for applications and tookits in the ubuntu archive that support it
• The Linaro Multimedia Team will work on consolidating the zero copy support for gstreamer and omax
• The Linaro Multimedia team will further optimize the ffmpeg AAC and VP8 software codecs for ARM such that it leverages NEON and multi-core support available in modern ARM hardware
• The Linaro Multimedia team will also support its own work as well as the multimedia software and hardware developer community by creating a multimedia test library that can be redistributed under a free software license

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/Multimedia

Performance

Desktopcouch speed and performance

• Slim down packages (ad-hoc couch-specific erlang build conflicting, libjs without xulrunner).
• Lightweight system-level proxy that handles deferred starting of couch, and multiplexing to individual per-user couches.
• Auto-compaction (opt-out).
• Investigate erlang's ability to have erlang processes limit cpu/memory use; tracemonkey (tracing), updated json serializer

NEON/Vectorizer Direction

• Work on multiple sizes first, then special loads and stores

• Investigate benchmarks and what could be vectorised, e.g. widening multiply (short[n] * short[n] -> int[n])

Performance inside GCC

• Continue with the current investigate/implement/repeat cycle
• Start measuring performance with CI and check for regressions

Improve boot-time performance on ARM, from power-on to user-space

• Unless you use lvm, cryptsetup, or raid, the kernel has all the information to be able to resolve an (fstype,uuid) pair itself! Fix the kernel to do this so we don't need to use initramfs at all
• Reduce the default timeout on u-boot to something less than 10 seconds

Performance improvement areas outside GCC

• Get GOLD on ARM up and running
• Add IFUNC support as the best way of handling A8/A9/NEON variants

Kernel Storage Performance

• SD and eMMC: Add support for background operations to drivers and to filesystems. Add support for reliable write. Investigate adding support for high priority interrupt.
• Add "trim" support to SD. Add to SD standard, implement in drivers, and appropriate filesystems.
• SD: Investigate eMMC (version 4.4 standard) "trim" support in Linux kernel and ARM SoC. Fill out implementation and fix bugs as needed.

Improve Memory Footprint for ARM

• Sample kernel memory usage as well as userspace

• Compare x86<->ARM (helpful to flag up arch-specific regressions)

• Discuss with the Ubuntu packaging folks about the footprint of update-manager, apt-xapian-index
• Define and sample some particular app use cases (web browser, media playback etc.)
• Follow up on the status of work on link-time dead code elimination, and hot/cold data partitioning and prelinking
• Analyse X resource consumption

Reducing Installation/CD Footprint

• Remove changelogs.Debian.gz (after a final legal check) [11 MB]
• Optimized PNGs and SVGs [12.5 MB]
• Remove perl and perl-modules (keep perl-base) [8 MB]
• Remove DRI MESA drivers for old cards, add Jockey handler [12 MB]
• If necessary, drop Evolution contacts sync from default install, which will drop couchdb/erlang
• Install footprint optimizations which don't affect CD size: compressed apt indexes, remove apt source package cache, remove rsyslog redundancy

Kernel Memory Regions

• Calculations showed 15% battery-lifetime extension from DRAM power-off use case.
• Additional use cases identified: having drivers announce contiguous memory-needs during boot, changing memory-region attributes at run time in order to compact non-movable memory.
• Paul to communicate use cases to IBMer working this to see if they can be accommodated.

Read further notes from this track at http://wiki.ubuntu.com/UDSProceedings/N/Performance

Hardware Compatibility

Key points and outcomes are summarised below.

• Improvements to X hardware support
  • Greater emphasis on handling unusual input devices transparently. This includes better support for unique/unusual buttons on unconventional form factor machines, hotkey support and the like.
  • Some commonly used devices can't presently be autodetected by X, explore using various techniques available to us to improve this - EDID, general quirks, device tree support ?
  • Enhance the gnome-monitors-applet to allow extra mode selection
• Refine the Kernel Bug Handling procedures.

  • Specific actions planned are detailed in the notes page for this track - http://wiki.ubuntu.com/UDSProceedings/N/HardwareCompatibility In brief a mixture of review and refinement of current practice and Wiki gardening to improve available documentation.

• Kernel Configuration and Ubuntu Kernel Delta Review
  • As is customary, reviews were undertaken of the kernel .config options and the delta between the Ubuntu kernel and mainline. Details of both are available in the notes page below.
• Stable Maintenance Release Schedule for the Kernel
  • The stable kernel team will change to a regular two-week release cadence. The cycle is two weeks long, and repeats every two weeks, with the exception of holidays, UDS weeks, and other considerations. An attempt will be made to sync the schedule with the needs of the Ubuntu platform schedule and Linaro release schedule.
  • A regular cadence will allow better determinism for community and OEM users of Ubuntu alike.

  • Specifics of the changes are detailed in http://wiki.ubuntu.com/UDSProceedings/N/HardwareCompatibility

• The Linaro team will be working on several hardware related tasks including
  • ARM Power Management
  • ARM BSP Managment
  • ARM Root file system on FLASH
  • ARM Kernel Standard Architecture
  • ARM Kernel Device Tree
  • Providing a hardfloat-enabled ARM port
  • Further details on each of the above in the track Notes
• Firmware test suite
  • The Firmware Test Suite is a tool that allows detailed low level testing and analysis of BIOS and ACPI codes on x86 systems. It is already gaining widespread support from hardware vendors and community users alike.
  • Various additions are planned to the suite, they are listed in detail in the track notes.
• Kernel Version and Flavours

  • Natty Kernel Version: Likely 2.6.38, the final decision to be made last week of Dec 2010. 2.6.38 is advantageous for linaro as there is an extra month to submit patches etc.

  • Natty Kernel Flavours
    • omap3 has been dropped at the moment in Natty, follow up investigation needed
    • Potential to have multiple ARM flavours
    • Discussion about supporting a flavour for 64bit kernels on 32-bit userspace concluded there was no compelling reason to do so, so the answer is NO.
    • low-latency will remain a community supported flavour.
• Hardware compatibility testing for Unity
  • Deliver 1st cut of detection-module to platform
  • Deliver 1st cut of application-from-hell to platform
  • Further detail in track notes.

Detailed notes from this track are at http://wiki.ubuntu.com/UDSProceedings/N/HardwareCompatibility

Other Notes

Detailed notes from content outside these tracks are at http://wiki.ubuntu.com/UDSProceedings/N/Other