##(see the SpecSpec for an explanation)

 * '''Launchpad Entry''': UbuntuSpec:server-maverick-uds-web20-workloads
 * '''Created''': 2010-05-24
 * '''Contributors''': ClintByrum
 * '''Packages affected''': 

== Summary ==

This specification outlines improvements to Maverick's support of common scalable web application (aka "Web 2.0") stacks such as memcache and gearman.

== Release Note ==

Better support for memcached is now included. The following packages are promoted to main: Memcache service (memcached),  language bindings for C (libmemcached2), PHP (php5-memcache, php5-memcached). Additionally there are new language bindings with libmemcached2 in universe for python, perl, and ruby.

gearman bindings for python, php, and java have been added to universe.

== Rationale ==

In order to assist users deploying scalable web applications on top of Ubuntu Server we should offer strong support for common backend server technologies. 

== User stories ==

 * As a WebOps developer I want to deploy memcache servers and memcache enabled applications quickly with a minimum of configuration and packaging effort, knowing that they will be supported through the life cycle of the product. I identify a need for caching, install on my development environment, and then roll out to production using identical packages.

 * As a WebOps developer I want to write my applications to use gearman to distribute load to backend servers and complete unrelated tasks without blocking. I use apt to install the appropriate language binding and gearman-job-server.

== Assumptions ==

n/a

== Design ==

=== memcached ===

==== memcached ====

The memcached server is extremely stable and in widespread use. It should be promoted to main.

==== language bindings ====

 * libmemcached2 - low level C library, considered the best client implementation
   * php5-memcached - promote to main, considered higher quality than php5-memcache
   * python-libmemcached - needs packaging for universe in maverick, main in later releases
   * libmemcached-ruby - needs packaging  for universe in maverick, main in later releases
   * libmemcached-libmemcached-perl - Memcached::libmemcached CPAN module, needs packaging, universe now, main later
     * libcache-memcached-libmemcached-perl - Cache::Memcached::libmemcached in CPAN, High level abstraction CPAN module, needs packaging, universe now, main later
 * non libmemcached based language bindings
   * php5-memcache - PECL module for memcache access (does not conflict with php5-memcached), popularity would suggest promoting to main even though php5-memcached is considered "better".
   * python-memcache - pure python memcache library - leave in universe
   * libcache-memcached-perl - pure python memcache library - leave in universe
   * libmemcache-client-ruby - pure ruby memcache library - leave in universe

=== gearman ===

gearman-job-server is already in universe. Perl bindings are also already in universe.

==== python-libgearman ====

pypi package. Package and add to universe.

==== php5-gearman ====

PECL module, add to universe.

==== gearman-java ====

Java client libraries for gearman, add to universe.

=== varnish ===

This will remain in universe. Latest version needs to be synced to deal with ESI bugs in 2.1.0 which was shipped with Lucid.

== Implementation ==

Please see: UbuntuSpec:server-maverick-uds-web20-workloads whiteboard for Work Items

== Test/Demo Plan ==

"This need not be added or completed until the specification is nearing beta."

== Unresolved issues ==

=== rabbitmq-stomp ===

The rabbitmq-stomp adapter is still considered experimental and is only available as a checkout from a mercurial server. We should produce packages when an official (even if just beta/alpha) release is made from the project.

== UDS Session agenda and discussion ==

Discussion plan:
 * Web accelerators:
    - varnish: All good!
 * Key/value store in RAM:
   * memcached (in universe dapper+):
     * promote to main: yes
     * Client libraries to promote to main: php5-memcache, python-memcache, libmemcached-tools, libcache-memcached-perl, libmemcache-client-ruby, libmemcached2.
     * only if based on libmemcached2, demote others
        * libmemcache (without the D) is stale & old - should be deleted
     * libmemcachedN is the right thing to package
       * libmemcached4 is coming
       * if we update, we need to rebuild   
     * Java client library to package in universe:
   * spymemcached
     * Package spymemcached in universe ?
       * not in debian either
     * compares how to Ehcache?
       * back Ehcache
 * web frameworks and their stacks (deployment tools):
    * ruby: RoR
      * TBD with lucas
    * python: 
      - django
        * already in main
      - pylons (in universe)
        * following debian, uptodate
      - turbogears (in universe, built on top of pylons)
        * uptodate, following debian
    * perl: catalyst: uptodate, follow debian
    * java
      * refer to seperate java session
    * php
      - symfony (1.0 in universe, deprecated upstream, clint already updating to 1.4)
      - zend framework (in universe, up to date) (we do have two versions: libzend-framework-php and zendframework)
      - cakephp (in universe, up to date)
 * Batch systems:
    * gearman (universe):
       * lucid 0.10, debian 0.12, latest 0.13
       * gearman-job-server is the current supported upstream
       * promotions/packaging only if they depend on libgearman
       TBD:
       - python client library
       - php client library
       - java client library
       - perl client library
 * Messaging:
    * AMQP: rabbitmq (main)
      * do we package the StOMP adapter? If not, do so
      * perl, ruby, python client libraries available
    * ZeroMQ (0MQ)
      * In Debian since somewhat recently
      * Ensure we're following debian
 * HA Proxy/Nginx/Lighttpd
  * Loadbalancing / director specialty
  * Security concerns?
  * Lighttpd
    * spare time project, community driven
    * ubuntu uptodate
  * HA proxy
    * no url rewrites currently
    * outperforms apache's mod_proxy
    * 1.3.22, maverick is up-to-date
  * Nginx:
    * async, very performant
    * closed development - no public source control repository, no security backports/test suite available
    * currently in universe at 0.7.65 (latest upstream)
 * Search/indexing:
   * Solr/Lucene (discussed in java stacks) - up-to-date in universe
   * sphinx popular in MySQL world
     * packaged as sphinxsearch in Universe 0.9.8 (0.9.9 current & in maverick)
   * Xapian (loosing market share).
 * Hadoop (discussed in separate session)


----
CategorySpec