Natty UDS Package Selection And System Defaults Proceedings

This page is here to collect together proceedings from sessions as part of the 'Package Selection And System Defaults' track at the Natty UDS in Orlando, Florida.

Please add proceedings by doing the following:

• Add a new section with the name of the session.
• Add the outcomes of the session as a collection of bullet points. The goal of the proceedings is to really focus on decided outcomes, so please try to keep them crisp.

Thanks!

Track Schedule

http://summit.ubuntu.com/uds-n/track/packageselection/

Monday - Proceedings

Kubuntu Natty Kontact

packageselection-kubuntu-n-kontact

• make sure kontact works from live image, currently akonadi shows error due to no running nepomuk startup, ask with upstream how to fix
• make sure lionmail gets packaged and works
• kolabsys can do testing, we will set dates, probably a week before beta and before RC to do that, takes two days to do their full testing, test can provide

Kmail 2 is NOT ready
Kmail 1 is working for some
IMAP is a special case
There is no scheduled release date for 2 (yet)
Split out Kmail2 for testing via a PPA?
Ship new KDEpim, or stay with the old? packaging problems either way
KDE-mobile -- still experimental, but on its way, we should package it (it's part of 4.5 tars we think so include with them)
It uses Akonadi, which Kmail does not, until version 2
make sure kontact works from live image, currently akonadi shows error due to no running nepomuk startup, ask with upstream how to fix
make sure lionmail gets packaged and works
kolabsys can do testing, we will set dates before Beta release and RC release

Kubuntu Packaging

packageselection-kubuntu-n-packaging

• ship with KDE SC 4.6
• ensure we have latest koffice, kdevelop, amarok etc
• inform debian of delta in packaging, highlighting parts they're interested in
• schedule session for wishlist for rekonq
• We don't want to ship a non-KDE browser, but Rekonq is not ready (and is horribly named)
• kwin Mobile will not be ready for Natty
• Go big with announce! Dot News, blogs, fridge
• still need to add KDM xsession
• Fix Cache issues using patches for kde4libs
• integrate automatic building with KDE dashboard
• make sure PPA packaging is based on current release as much as possible
• (re-)propose to tech board to include updates in -updates, QA'd in PPA first
• we need to test ourselves, since we cannot rely on upstream for that (or good change documentation
• get 4.4.5 into lucid (kontact 4.4.7)
• turn on raster for plasma for an alpha
• try likeback in alphas
• implement likeback in kdelibs and turn on for all apps for alpha/beta
• switch from qtcurve to gtk-oxygen-engine, ensure settings update properly

Arm Server Optimized Lamp Stack

packageselection-arm-server-optimized-lamp-stack

Problem

• No solid benchmarks for server apps on ARM or collected data can't be shared
  • So what does it mean to have a high performance LAMP stack on ARM?

Questions

• What kind of utilities and tools can we use to benchmark a system?
• What type of packages do we need to provide a high performing LAMP stack on ARM?

Goals

• Figure out the benchmark numbers as they are for ARM, compare to Intel/AMD, then improve.

Benchmark Candidates

• Pulled from the Linux Test Project test tool table (http://ltp.sourceforge.net/tooltable.php)

  • Hammerhead is a web server stress tool that can simulate multiple connections and users.

    • http://sourceforge.net/projects/hammerhead/

  • httperf is a popular web server benchmark tool for measuring web server performance

    • http://www.hpl.hp.com/research/linux/httperf/

  • Siege is an http regression testing and benchmarking utility.

    • http://www.joedog.org/index/siege-home

  • PagePoker for loadtesting and benchmarking web servers

    • http://node.to/hacks/src/

• Java

  • VolanoMark lets you determine the performance and connection limitations of your hardware, operating system, and Java platform when running the VOLANO chat server.

    • http://www.volano.com/benchmarks.html

• Phoronix - http://www.phoronix-test-suite.com/ provides an extensible framework for which new tests can be easily added. It's designed to effectively carry out both qualitative and quantitative benchmarks in a clean, reproducible, and easy-to-use manner.

• Others - Webstone, Bonnie (want to test CPU not AHCI cards etc...), LMBench, Sysstat

Other ARM Server workloads

• Squid
• MariaDB/Drizzle

• Redis - http://code.google.com/p/redis/

• Memcached

• HipHop for PHP

• PHP workloads (with benchmarks?) Drupal (http://drupal.org/node/79237), Wordpress (has benchmark)

• Samba (?)
• Cluster file systems (?) GFS / Gluster
• Shark JIT on ARM
• Python, Twisted (used by Launchpad), Ruby on rails (new kid on block but growing)
• DNS - bind (Memcached a good proxy for DNS performance - essentially a look up)
• HAProxy - user space - interesting

Testing

• Look at adding hooks into checkbox for benchmark testing on ARM
• Martin B, Dave Rusling (Linaro), Chuck, Leo (ARM) - willing to help with testing with hardware and appropriate information that makes things easy to run
  • Scripts? Apt install on top of Linaro testing?
• Results to be hosted on UDS wiki for now
• Recomended kernel config

  • + perf / oprofile
    - debug
    + large block handling
    

Profiling

• Dave Martin / Will Deacon (ARM) will post pointers to info about using the Linux perf tools to collect profiling data.
• Important to collect profiling information during benchmark runs

Server root fs

• How to grab server root fs - grab right ARM image - no server ISO right now
• Martin Bogomolni to test root stack / media create to base file system only

  • - Exact binaries needed for testing
    - Align with Linaro methodology on this
    

Next Steps

• Distribute work
• Collect results (once testing made easy / repeatable)
• Circle back on identification of focus areas
• Identify bugs / route to owners for resolution (missing packages, broken, poor performance etc...)

Finish Upstart

packageselection-foundations-n-finish-upstart
Last UDS (recap): Stable version of Upstart in Ubuntu now for a few releases, working out reasonably well, but there are a number of things we need to fix (the fact that mountall is needed, user services, etc.). Used last UDS to get a sense of what complaints were from various people and groups, and made sure that appropriate bugs were filed.

Upstart's design is too simple. The goal of the next version is to fix problems based on deployment experience while retaining the "Upstartishness", and reach an elegant, simple design that we don't need to change again: i.e. 1.0.

https://bugs.edge.launchpad.net/upstart/+bugs
https://bugs.edge.launchpad.net/upstart/+bug/447654

Proposed Changes:

• add the concepts of states which are based on events and persist beyond the event, which jobs can depend on
• child tracking should use proc connector
• events should be queued against jobs when the job is transitioning
• overrides to local configuration of jobs without editing them
• new hook on starting * to allow tracking of job dependancies
• upstart will know about chroots and make itself available in there if it is going to start jobs automatically; and use the root directory for local start within a chroot to start the right jobs

Need to get buy in for these features

GNOME plans for the natty cycle

packageselection-desktop-n-gnome3

GLIB 2.28

• Ready by December
• Non-controvercial release
• Still a number of applications require porting to gsettings
  • gnome-panel in process of porting
  • We should generate a list of applications Ubuntu needs porting to have the CD gconf-less

GTK3

• Plan of record to release in December
• There will be more ABI breaks coming before then
• Will need both versions on the CD
• List of libraries in jhbuild that will need a GTK2 and a GTK3 library
• We only want gsettings/GTK3 in the LTS
• No GTK# bindings for GTK3

  • Non-trivial amount of work (see: https://docs.google.com/document/pub?id=1zYYQTo148TQOFLF2y1IzKSMsLJJTcWJkYhXbZNhVgHs - notes from the GUADEC Banshee BoF which touches upon this, aaron bockover has possibly started work on gobject-introspection for .NET, someone should contact him)

GNOME3

• Clean the CD this cycle of obsolete libraries:
  • gnomeVFS, libglade, libgnomeui...
• Upstream needs feedback on gobject-introspection/PyGTK
  • port our apps (software-center, apport, etc.)
• GNOME Control Center etc are all or nothing upgrades
  • Rodrigo has a plugin that provides the old gconf API for older apps
• GNOME Shell will be available in Universe
  • Ubuntu patched apps don't always work perfectly (e.g. indicators) in different shells
    • Need a consistent way for these patches to detect they are in vanilla GNOME and behave as upstream intended (ryan)

Package selection

• Drop tsclient, and use vinagre if it supports RDP, otherwise consider Remmina
• Consider sticking with existing versions if packages are doing risky changes (like we did for Maverick)
  • Most packages should be OK, nothing controvercial
  • Keep evolution at 2.32 as of now (will be updated the week after UDS), see what's going on later on the cycle.
  • Totem should be fine for 3.0 (vuntz) - it didn't make a 2.32 release
  • Consider replacing EOG with shotwell (fagan via irc) - need to ask shotwell devs, for Maverick they didn't recommend this
  • Nothing major happening in GNOME Games, gcalctool
  • GEdit is gold as always
  • Cheese has new features but should be good (vuntz)
  • Yelp can go to latest version now (changes were held back)
  • GDM update to 2.32. GNOME Design has mockups for new greeter, but wont be ready for 3.0
  • Applets may not work with the latest gnome-panel
    • Tomboy might need porting
• Can get rid of gnome-system-tools?
  • Only thing being used is time configuration
  • Features are being merged into gnome-control-center
• gnome-utils
  • Package has been split since maverick
  • Drop gnome-dictionary - doesn't work outside of english, not well supported
  • gnome-search-tool - not needed in Unity, needed for 2D experience
  • an environment variable to add to get some default GNOME experience and don't break the GS

Server Application Packaging and Distribution

packageselection-server-n-packaging-and-distribution

Server (and specifically Java) applications present a challenge in terms of packaging for Ubuntu. Alternative packaging and distribution methods should be considered to ensure that Ubuntu can support server application requirements whilst minimising overheads.

Topics for discussion

• Policy of when server applications and services should sit outside of the normal release cycle
  • Contributing factors
    • Frequency of upstream
    • Complexity of dependencies/overlap with Ubuntu
    • Demand for application or service
    • Plans for build-from-source packaging
• Where upstream development methodology does not fit with build from source
  • Java stack packaging approach
    • Build from source or binary packaging?
• Distribution methodology
  • Options:
    • PPA - have all the functionality necessary already to group packages and dependencies to ride on top of the core releases.
    • Upstream conduits?
  • Challenges: visibility from update-manager/apt
• Roles in packaging and distribution
• Other Considerations
  • Security vulnerability monitoring and patching
  • SRU equivalent processes.
• Examples to support discussion from Maverick
  • Cassandra
  • Hadoop
  • Terracotta
• Potential candidates for Natty
  • Hudson
  • JEE Application Servers
  • + Others

Session notes

  * Server applications specifically
  * Many projects release early, release often, and that isn't a good fit
    for the distribution cycle

Ideas:

  * Conduits - blessed PPAs (blessed by Ubuntu or the upstream project)
    - e.g. drizzle
  * Added specific pakcages (or specific versions of those packages), while
    not changing the base OS
  * Sometimes libraries are added to these PPAs, which can cause problems
    with other applications. SONAMEs mitigate this somewhat, but aren't
    perfect, and there are also cases where problems can be caused by
    behaviour changes in new versions etc.
  * Use copies of libraries if you need an update, rather than changing the
    version used by everything on the system?
    - Harder for sysadmins to know what is on their system.
  * Something people are doing now (mathiaz - 95% comes from archive, 5%
    from upstream) - making it easier and encouraging best practices.
    - Spamaps: more aiming for fast-moving projects: Cassandra et. al.
    - ttx: two related cases we are dealing with Cassandra, and
      getting latest packages for the area that you care about.
  * ScottK: clamav may provide an example we could follow, maintaining
    current clamav releases in all supported Ubuntu releases. Assuming that
    we can't do it in the archive may be a mistake.
  * backports might be useful to, if the bug is fixed that means you get
    everything in backports if you enable it for everything.
  * Dustin has had success with pushing to PPAs for all releases when he makes
    a release of his projects.
  * mtaylor: leaving old versions in the archive still gives you a headache,
    as that's where people go by default.
  * ttx: possibly two solutions. "Freshness" could be fixed with backports,
    should talk about "unpackageable" packages.
  * Java is a common source of these packages.
    - Getting everything built from source and packaged is a huge task given
      the number of libraries required by the transitive dependencies.
  * ttx: is allowing distribution of binary versions of these packages a solution?
  * slangasek: what value are we adding if we are just passing on the upstream
    binaries?
  * mathiaz: discoverability: apt-get install hudson
  * ttx: some level of integration, e.g. deploying hudson with tomcat
  * SpamapS: did it with Cassandra: have a PPA which uses upstream packages,
    based on binary jars embedded in the package.
  * ScottK: what difference to multiverse?
  * multiverse is a definite possibility, and wouldn't really be the first time
    that we would have used it for this.
  * SpamapS: upstream are releasing these things in a format that works for them,
    maybe we are missing a tool that allows us to package them easily.
  * multiverse implies a value judgement on the package, so could take care
    to document why they are in multiverse.
  * A related problem is that upstreams will demand particular versions of packages.
    Providing multiple versions in the platform might be needed.
  * Upstreams often see the platform as constraining, so bundle their libraries
    so that they can have greater control.
  * We usually do that for security update purposes, and that is some of the value
    that we add, so there's a fine line to walk between enabling upstreams
    and providing security support.
  * Some nascent effects to encourage Java developers to use common versions.
    - maven makes it a hard sell.
  * cjwatson: get them to ship test suites, so we can have some confidence that
    if we change library versions an app uses it will work.
  * mtaylor: as much as I agree with our approach, convincing these communities
    may be tough when they are heading in a different direction, and other platforms
    don't follow our model.
  * ttx: for each of the "unpackageable" things we should look at the cost of
    doing it right, before using any alternative method.
  * SpamapS: if we aren't adding value, we should get out of people's way
  * ScottK: is the integration ttx described enough value to make it
    worthwhile chasing this?
    - not really
  * slangasek: if we don't support it then don't make it look like we do.
  * mtaylor: probably more upstreams that would create a PPA. Better tools for
    packaging could help with those that don't know how to package, even if
    the packages are crappy.
  * Documentation problem. It's fairly easy if you use the right tools.
  * ttx: based on the amount of added value multiverse would make sense, with
    PPAs where we don't add any value?
    - yes(?)
  * james_westby: build tools for upstream development communities to generate a debian source package.
    
    * Make creating source packages OBVIOUS and SIMPLE
      - "debuild" without ./debian
      
  * Dual approach: go multiverse+backports or PPAs based on Ubuntu-specific added value
  
  * To be continued in a "discoverability" and a "language-oriented dev tools" sessions

Ubuntu Chinese Edition

packageselection-desktop-n-chinese-version

• Outcome
• Outcome
• Outcome

Integrate Wine into Software Center

packageselection-n-desktop-wine-software-center

• Have applications offered in the Software Centre
• Direct X Support could be marketed as a potential benefit
• Could use a windicator for the user to notice that he is running a Windows program
• Nothing today to prevent Wine from making general system calls, so we need to find a way to isolate things to prevent accidental or malicious use.
• We can handle prefixes for packaged cases

• Show apps that have been installed in the /home directory.

  • Could remove using Software Centre even though they were installed by other means
• We should promote our own applications when the user is installing a Windows program

  • for example, if a user wants to install MS Office, we warn him that LibreOffice offers the same functionality

• In a multi-user case the removing of things may be difficult.
• Cautious Launcher isn't favored because it's not easily translatable.
• Channel thread that we should encourage local Linux apps rather than enable old Windows addiction
  • Adding to the above point, if Canonical notices that a Windows application gets lot of users, Canonical should encourage the application developers to port it to Linux, or even propose to port it themselves.
• If we can make things smoother it will lead to more Ubuntu users.
• We need to be sensitive to GPL redistribution issues
• From a commercial perspective, could we use this to enable "Windows only" program to get it in the Software Centre.
• The use case of getting Software Centre to list the applications and remove installed apps is something that would need to be written.
  • Designed first, then code written

Bash Tab Completion and command-not-found enhancements

packageselection-server-n-commandline-userfriendly

Main topics

• bash argument completion
  • lack of consistency (sqlite doesn't let you complete database files)
• command-not-found
• Bash tab completion unpredictable, handled differently by each different program

bash-completion

1. identify packages without completion information
2. review current ones and make sure they're accurate
3. file bugs accordingly
4. Investigate Apport Integration

command not found

• support arbitrary hints
• ACTION: set opinionated defaults for the server seed (put in a special package)
• ACTION: modify command-not-found to use ^
• ACTION: publicize this capability to ubuntu / debian developers

Tuesday - Proceedings

Photo printing is not user-friendly and often broken

packageselection-desktop-n-improve-photo-printing

• Outcome
• Outcome
• Outcome

General networking stack enhancements

packageselection-n-network-stack

• Outcome
• Outcome
• Outcome

Session Name

UbuntuSpec:

• Outcome
• Outcome
• Outcome