#ubuntu-server on Libera Chat IRC |
||||||||
Introduction
This page is specifically for the gathering of proposed features for the Oneiric (11.10) development cycle. Prior to UDS, we organized the proposed Ubuntu Server track sessions, based on a subset of the proposed features. The list below represents each Ubuntu Server track session and the notes taken during the associated discussion.
UDS Discussion/Feature CANDIDATES
Notable Decisions and Planned Changes
Best OS for the Cloud
- Adding ARM as a supported architecture
OpenStack will be the default cloud infrastructure stack for Ubuntu Server, replacing Eucalyptus
Both OpenStack "Diablo" and Eucalyptus 3.0 will be in main
- Moving to a 2-stage install experience
- Minimal set of steps/packages required to boot a Server on CD
- Optional configuration/deployment via Orchestra components
Ubuntu Enterprise Cloud is now Ubuntu Cloud
- Detailed boot messaging enabled by default, i.e. remove "quiet" from default grub options
- We will replace ntpdate with ntpd in Ubuntu Server
Best OS in the Cloud
- Ensemble will be in universe
- byobu on by default in cloud images
Session Blueprints
Blueprint: server-o-arm-server
Action Items
- Look at adding shipping seed to the headless image - if the server is not on a network, won't install because there is no shipping seed
- Investigate hot plug support
Use OpenStack for ARM + LXC
- OEM-config runs on the serial console, needs a switch to appear on serial AND/OR console
- Select platform for ARM server release:
- OMAP3 beagleboard XM
- OMAP4 pandaboard
- Look at ARM Versatile Express
- Preseed, vs unpack and boot (which format)
- Have a proper debian installer
- Need to have both netboot + preinstall (headless) + bootfloppy
- 'bootfloppy' has different definition based on platform
- JAVA - dependencies on openJDK, getting java working better on ARM
- ARM java VM's are optimized for client, need to be optimised for enterprise/server
- IST project has ARM support
- Investigate Landscape/Smart issues
- Network filesystems for diskless boot must be tested and supported - NFS (with and without RAMdisK), iSCSI, NDB, AFS(?), CEPH, GFS2, PVFS, RAID, LVM
- Orchestra tie-ins will need to be tested
- Investigate the ability to support bonding and bridging networking (TUN/TAP/802.1q/VLAN/VPN/IPv6/IPsec)
- Investigate interpreted/compiled languages that have Server dependencies:
- Erlang
- python
- Ruby (Rails?)
- Automated testing on ARM platforms needs to be done.
- Server team test cases need to be looked at.
- LAMP stack testing
- Crypto acceleration in kernel
- urandom, random, random number
- entropy keys
- 32 vs 64 bit
- gnuTLS, SSL, etc need to be validated
AppArmor on ARM, it's not validated yet on ARM
- toolchain, check that hardening is being called (Case)
- memory randomization
- toolchain, 64 bit atomics are missing in GCC 4.5
- System Management Tools!!
- Net install needs works on OMAP4
- Calxeda will provide a PXE boot patch for u-boot
Blueprint: server-o-boot-experience
Action Items
- [cjwatson] turn off quiet for server
- [jamesodhunt] allow upstart visualization tool to filter out jobs/events that aren't in existence
- [tbd]define/document a simple job/alias/event for "local filesystems are up", for local services
- [tbd]define/document a simple job/alias/event for "networking up", for generic network services
- [tbd]define/document a simple job/alias/event for "network filesystems are up", for network filesystem services
- [tbd]define/document a simple job/alias/event for "system has booted", for /etc/rc.local type behavior
- [tbd]need to improve start on for cron (start when all user info available)
- [tbd]update Upstart Cookbook with "abstract jobs" and full exposition of how to define optimal "start on" and "stop on" for a new job
- [clint-fewbar] file bugs on services that don't respect single user mode appropriately
- [clint-fewbar] automated testing for single to multiuser
- [clint-fewbar] add init-checkconf rule to lintian for upstart jobs need to measure boot speed difference between sourcing /etc/default files, and not
Blueprint: server-o-byobu-cloud
Action Items
- Enable disabling all status monitoring eye candy easily
- Easily enable/disable byobu in metadata
- Easily disable byobu
- MOTD messaging
- respect an ssh environment variable locally on the ssh client that says "no byobu" to the remote server
- roll Alpha1-3 images with Byobu enabled, revisit decision before feature freeze with plan of including in 11.10 barring any critical regressions or issues
Blueprint: server-o-ceph
Action Items
- put it in our automated testing suite first to make sure it works as a fs before going further
- also test the tools
- investigate qemu-rbd
- investigate libvirt
Blueprint: server-o-cloud-images
Action Items
- add chef support (like puppet code)
- fix cloud-init upstart jobs for lxc
rename uec-images -> cloud-images.ubuntu.com
- what about "ucloud-images" ?
- and, of course, add a redirect, right?
- udev: start on virtual-filesystems
- networking.conf: start on (local-filesystems and stopped udevtrigger)
- cloud-init-local: start on mounted MOUNTPOINT=/
- [smoser] push on vmware help a bit for uec-images.
- [smoser] read about ovf 2.0 ...
- improve documentation
- make sure images are usable by any OVF consuming tool
- modify the disk format to be something non-VMDK, or improve the linux support for VMDK images (kvm).
- cloud-init
- support configuration of networking via cloud-config this would be useful in OVF or other places, where the metadata comes from a non-network source.
- do better if there is no datasource available, and do not require one to always be there
- uec-persistency (bridge gap to lack of EBS root)
Both OpenStack and Eucalyptus 3.0 will support EBS root, so no need to publsh UEC-peristency kernel images
Blueprint: server-o-cloud-migration
Action Items
- Address the following migration issues:
- Users
- Groups
- Vols
- Instances - with virsh / config?
- Images
- Buckets
- Snapshots
- Permissions/credentials
- Keypairs
- Security Groups
- Elastic/internal? IPs
- Quotas?
- User Groups
- Usage
- Support script migration directly from euca to openstack.
Blueprint: server-o-cluster-stack
Action Items
- Move forward with OCFS2 as the supported clustered FS
- Keep RHCS 3.0 for DLM's only.
- dlm_controld.pcmk/gfs_controld.pcmk
- keep shipping gfs2-tools (and make sure it's ok)
- upgrade to latest 3.0.x that contains .pcmk (3.0.17)
- Package fence-agents/resource-agents
- Upgrade pacemaker to 1.1
- Investigate and resolve ARM related issues within OCFS2, test
- Move from cluster-agents to resouce-agents
- Link up with openstack and upstream to compile a list of HA requirements
- DRBD 8.4 if is shipped in mainline kernel Oneiric ships.
- ***Documentation***
- Sort out PPAs (ubuntu-ha-maintainers becomes main PPA, all other go away)
Blueprint: server-o-cobbler-next-steps
Action Items
- Integration with puppet - ensure works in Debian/Ubuntu
- Meta-data service
- cloud-init support requirement
- look at inventory service from puppet
- User Interface
- rebrand through CSS for Ubuntu
- implementing new features in webui need to happen upstream
- gpxe support in cobbler
- snapshot of running systems to generate profiles?
- orchestra features rather than cobbler
blueprint (DevStructure) might fit this
- package manifest grab NOT preserve data and state.
- koan
- support for kvm instances today.
- features need testing and improving as required.
- feedback to upstream ubuntu specific changes to cobbler for better support on multiple distros:
- DNS integration
- DHCP integration
- Better support for detection of distributions.
- Validate desktop deployment testing through cobbler.
- Power/infrastructure management; discuss in power infrastructure session.
- [martinbogo] requirements for supporting uboot through cobbler.
Blueprint: server-o-complex-deployment-testing
Action Items
- Find a base framework to support boot and deployment testing:
- Boot:
- Minimal install - startup and shutdown OK
- Full install - startup and shutdown OK
- Complete upgrade and then re-test.
- collect metrics on boot time
- Image Proving
- RAID testing.
- Multipath and encryption
- ISO installs of AMI
- Package Removal Testing:
- install services
- start services
- check services
- stop services
- uninstall services
- re-install same services
start & re-check services
- Boot:
- Configure for complex deployment scenarios:
- Wiki + Database + Memcache + Load Balancer
- Bridging and bonding of network interfaces
- Networking File Systems
- Openstack Deployment test
- Launchpad
- Samba + LDAP or Kerberos Backend
- Backup solutions
Blueprint: server-o-conf-management
Action Items
- noop dpkg hook for configuration file so we can ignore conf files on a per-file basis, ie: dpkg --ignore-conflicts-to /etc/path/to/some.conf
- suggest as a best practice policy for upstream server projects to support a conf.d style directory structure
Blueprint: server-o-documentation
Action Items
- Jim to look at build toolchain transition needs to produce something similar to docs.openstack.org
- Look at what we currently have, what needs to change, and plan how we do it
- Robbie to look at getting funding for dedicated document resource
- Jim or Adam to look at an Ubuntu Open Week-type session regarding additional contributions to server docs
- Server team to identify server resources that could be used to test out server documentation configurations for the purpose of documenting them.
Blueprint: server-o-drop-vmbuilder
Action Items
- [davewalker] Investigate CCA requirement for vmbuilder
[smoser] change uec image build -> use live-build
- discuss with cody what is appropriate release to target for 11.10
- existing web/wiki documentation would need to be updated to at least mention
- [nijiba] if decision is to drop support for vmbuilder, do it quickly, mark it dead. let it be forked. talk to canonical legal
- assuming above, add deprecated warning on vm-builder
- blog post and post ot mailing list with result of this stating we are in need of maintainer
Blueprint: server-o-ecryptfs-testsuite
Action Items
- john going to take another crack at long filenames this cycle
- rtg not opposed to carrying this if we have to
- has a fallback mode, maybe first 8 chars are real filename, dash, md5sum
- backward compatibility
- rtg says gotta have xattrs to do this elegantly
- jj will refresh the xattrs patches soon (next few weeks)
- kirkland will look up the re-encrypt-all-files script he wrote for a previous ecryptfs cve
- kirkland will write some basic mount/write/unmount/read generic files (random binary, text, small, big) (take from ecryptfs-setup-private), test failure cases too (long file names, bad data in underlying filesystem)
- check phoronix (or LTP) for ecryptfs stress tests
Blueprint: server-o-ensemble
Action Items
- package for universe and determine how to deal with updates post 11.10 release
- [niemeyer] Ensemble Daily PPA.
- package txzookeeper
- upgrade txaws
- update zookeeper with patches already pushed upstream
- determine if we need to package principia, and do so, if need be
Blueprint: server-o-ensemble-orchestra-openstack-support
Action Items
- gustavo to send soren email regarding needed api functions for twisted client lib
- cloud init to support Rackspace meta-data (nice, not required)
- orchestra to give a twisted API to create physical machine that ensemble can call (aws or openstack)
- have a simple way for orchestra to deploy a pool of machines that are made availabe for ensemble
Blueprint: server-o-eucalyptus
Action Items
- work with Eucalyptus to update packages from first code drop expected mid June.
- continue re-syncing and testing through GA release by August 15
- continue to test and file bugs after GA
Blueprint: server-o-hadoop
Action Items
- [james-page] Review packaging to date and work with iamfuzz
- [kirkland] Discuss with Cloudera the build/runtime dependencies with [oracle|sun]jdk
- work on how we can add visibility to modules in orchestra in order to determine if we need to work on putting CDH in Universe or main
- who controls the list?
- what rules to accept new items?
- how is it published?
Blueprint: server-o-infra-power
Action Items
- write psu driver
- do work for puppet integration (vs orchestra / cobbler...)
- package fence-agents (new package will land in Oneiric - separated from cman)
- create PowerNAP interface (broker vs basic use)
- create a tool to find USB and network devices (nut-scanner)
- add avahi support to NUT, and allow easy discovery of power infrastructure
- create a library and CLI/GUI (nut-config)
- add augeas support (create augeas lenses)
- create a udev helper (a la hal-cups)?
- decide on what to do with the web UI (CGI improvements or full web 2.0 revamp?)
- make NUT-Monitor improvements, including device discovery and configuration
- create native upstart / systemd scripts
- create provisioning recipes
- distribute the new documentation (nut-doc-html and nut-doc-pdf)
- monitor remaining time before shutdown in upsmon
- provide scriptlets, for common actions (send mail, interact with VM, ...)
add support for UPS and PDU outlets on/off/cycle through NUT to PowerNap
- add NUT support (avahi discovery + libupsclient) to UPower
- add shutoff support (UPS poweroff after a power failure shutdown) to UPower
- create a NUT bridge (NUT drivers that report in UPower namespace)
Blueprint: server-o-install-experience
Action Items
- [cjwatson] Add support to partition a system across multiple disks at install, in a non interactive, automated manner
- Support the ability to run Ubuntu only from SAN, ignoring altogether local storage
- [Boris] provide cjwatson with a preseed/kickstart file of 'what he wants to work'
- [kirkland] Support kernel command option to run d-i inside of byobu/screen to allow for remote attach/detach, real time monitoring of IO (disk throughput, network bandwidth), system load, stats/notes about installation
- martin to send cjwatson details on OpenIPMI : Serial over LAN for console during install (device name may not be tty[S]0. ttyam0)
support network bonding -> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=611250
- Support MTD partitioning for ARM server: Debian GSoC project from a year or two ago - should be possible to retrieve that and polish it up
- write a u-boot-installer udeb to support install of a bootloader other than GRUB (u-boot for ARM server)
- verify consistent network device naming support works as required when the server is booted with biosdevname=1
Blueprint: server-o-jasmine
Action Items
- package JASMINe
Blueprint: server-o-jenkins
Action Items
By Alpha 1:
- [james-page] upgrade to latest stable branch from Jenkins + upgrade dependencies
- [james-page] Resolve outstanding build-from-source issues in dependency chain
- [james-page] Prepare library packaging for upload to Ubuntu
- [james-page] Sponsored upload of jenkins plus dependencies to Ubuntu Universe
By Alpha 2:
- [james-page] look at JVM compatibility with xstream (for example)
Additional Work Items:
- [james-page] feedback libraries to Debian through debian-java team
- [james-page] re-sync libraries from Debian as a when required
Blueprint: server-o-jonas
Action Items
By Alpha 2:
- [james-page] Review current OSGi packaging
- [james-page] Work with florent on understanding JOnAS clustering and how this might feed into Ensemble and Orchestra
- [james-page] Maven3 key for delivery - work upstream in Debian to help with Maven 3
- [fbenoit] Identify priority list of ow2 components for packaging during Oneiric
- [fbenoit] Idnetify list of ow2 and other maven plugins required to support JOnAS
Blueprint: server-o-kvm-document-suggested-changes
Action Items
- Define workloads that we care about
- Improve documentation on good settings for these workloads
- add a small section to the server documentation
Kernel and Network tunables best practices / recommendations to be documented (Classic performance tunning for background workloads) => link to existing server doc on tunables
- Make decision about not having swap in guest
- Make decision about bridging over bonding to be tested and documented
Cgroup to be tested and documented (should work => user space)
Blueprint: server-o-load-testing
Action Items
- Get a list of real world stress tests.
- Another request is that the test be packaged so external parties can run these load tests.
- Look at having a reporting/management tool to segregate/differentiate between hardware configurations.
- Have small rescue env booted automatically (from grub, think memtest for server hw]
- [hggdh] asking someone to package basic "ubuntu-test". which would include these basic tests.
Blueprint: server-o-lxc-improvements
Action Items
- quick container cloning with snapshotting for lvm, btrfs, and aufs/unionfs (based on ppa:serge-hallyn/lxc-natty work)
- sandbox template using current user namespace
- per container syslog
- proc virtualization
- audit (LAF) virtualization (hallyn move this where you see priority)
- container reboot and shutdown
- custom per-container firewall rules
- lsm integration
- apparmor integration
- templates for other distros (fedora, opensles, gentoo)
- sandboxing with seccomp2
- zul: List of missing bits pieces of libvirt for lxc (libvirt 0.9.1-1 merge is pending)
Blueprint: server-o-lxc-sandboxing
Action Items
- [jjohansen] kernel PPA
- [hallyn] custom lxc in ppa with extended configuration to drop syscalls. (add to lxc sprint agenda)
- [ubuntu-security] [hallyn] new 'sandbox' program which generically wraps any program in sandbox.
- compare this to what stgraber has developed for application sandboxing (other session)
Blueprint: server-o-monitoring
Action Items
- Write an mcollective plugin for nagios
- Change nagios to do the triggering
- Check to see the services to monitor
Blueprint: server-o-mysql
Action Items
- [clint-fewbar] package MySQL 5.5 for Debian and Ubuntu
- [clint-fewbar] package Percona Server for Debian and Ubuntu
- [clint-fewbar] package MariaDB for Debian and Ubuntu
Blueprint: server-o-opengrok
Action Items
[jamepage] identify any outstanding licensing issues around OpenGrok and dependencies
[jamespage]: Package remaining OpenGrok dependencies and into a PPA
- Speak to IS about storage, cpu and deployment
- Speak to Design Team about how to handle the branding aspect of the web interface
- [jamesodhunt] Investigate structure of repository and alignment to Ubuntu
- Determine if it interfaces with BZR
- Determinie if any additional languages would need to be supported (by writing ctags parsers)
Blueprint: server-o-openstack
Action Items
- [ttx] share slides "Nova modularity" (and updated)
- rackspace considered "best practices" also.
- [openstack] provide a list of questions
- database backed
- vlan / flat networking
- [ubuntu/canonical] need to define the components that are used (mysql/postgres/sqlite, vlan segregation v. flat networking)
- kvm/xen/hyperv...
- [ubuntu/canonical] define 2 "topologies" to target
small developer install (1 machine -> 2 machines)
- recommended larger install (may take 3,4,5+ machines)
nova, glance & swift integration
- puppet recpies
- orchestra integration
- create ec2 testcases (to verify ec2 compatibility)
- figure out what installation look like (ie, "uec install")
Blueprint: server-o-orchestra
Action Items
- investigate adding gpxe support in cobbler
- review cloud-init, as it might need some changes that make it more consumable here
- It might be useful to allow at the cobbler (or provisioning level) user-data that then would get to cloud-init.
- somewhat obvious thought... would potentially be nice if there was an ec2-like api cobbler/orchestra
- cloud-init should be treated as the "configuration management bootloader", i.e., the gateway to being managed by puppet/chef/ensemble/rightscale/etc
- initial pxe boot image on the network should be a "collect data and report it" image. Then, when a new system is plugged into the network it will boot that image, and report back its mac and cpu/meminfo/disk...
- support Ensemble on bare metal
- they will need a machine installed with Ubuntu (somehow) and the agent running
- also needs an orchestra api to trigger installations (run machine)
- pre-execution environment ( boot, collect data about system, report to cobbler )
- [davewalker] discovery mode (facter) collect system information
- hardware configuration
- test squid deb proxy pre-existance (scenario with 2 servers)
- add wrapper for cobbler that works/looks/feels/acts like the ec2 api
- [teyo] will help with OpenStack/Puppet modules
- setup automated testing
- [clint] get a feel for deploying ensemble on orchestra
- if this is doable, ensemble and puppet should be interchangeable modules, with ensemble as the default to encourage more formula composition [robbie.w]
Blueprint: server-o-paas-stacks
Action Items
- Package VMWare's cloudfoundry
- Package Google's appscale
Blueprint: server-o-personal-cloud
Action Items
size and scope what's involved in a U1 client working on server (can lpad (http://goneat.org/lp/lpad) be used for this?)
- investigate Diaspora
- anything that can be done to facilitate small office site-local ubuntu one servers
Blueprint: server-o-postfix-enhancements
Action Items
- [lamont] multinstance support
- [spamaps] upstart documentation
- [spamaps] apport pattern for invalid hostname
- [lamont] not preseedable - investigate if this is fixable and document it either way
- [lamont] sqlite support
- [spamaps] No backscatter by default on new install - investigate
- [dannf] - unchroot script
- [ivoks] - mail stack deployments via puppet, etc.
- [ivoks] - try really hard maintain mail-stack-* packages
- [kitterman] clean up deprecated packaging changes
Blueprint: server-o-powernap-improvements
Action Items
- Define sample profiles i.e. For Web Servers, File Servers, Laptops etc
- Perhaps involve someone on the Desktop Team to improve powernap for Laptops
Blueprint: server-o-puppet-integration
Action Items
- Add Tasksel options ( server ) to puppet
- Create Orchestra recipes for:
- rsyslog
- nagios
- cobbler?
- bzr/git
- fs creation/management
- jenkins (master and slave)
- ha cluster (drdb, pacemaker, corosync, cman)
- kvm (host and guest?)
- LXC
- Xen
- investigate puppet faces
- Report puppet configuration ( no .d )
- Investigate methods for helper script for puppet modules install
- Puppet dashboard packaging from upstream for inclusion in Ubuntu.
- Puppet 2.7 for inclusion in Natty.
Blueprint: server-o-remote-desktop
Action Items
- [soren] to package noVNC
- Investigate how important is sound
- launching
- use cases
- desktop demo
- cloud server admin who just wants a damn desktop
- MIR server/client packages (if not already in Main)
- do some real testing of VNC to AWS (from various clients) in comparison to NX
- would like for cloud/vm/remote-desktop style hardware to be tested by the Ubuntu Desktop team as a valid "hardware platform"
- support some mechanism for downloading arch or os specific info
- support configuration via applet parameters
- download via http or https
- download a list of files, from different sources
Blueprint: server-o-seed-review
Action Items
******** F I X M E ********
Blueprint: server-o-split-cloud
Action Items
* minimal install disk * dropping tasksel and UEC(under condition) * put what is needed to have a bootstrapping server and mirror
Blueprint: server-o-sru-process-review
Action Items
- Do a rotation of SRU reviewers like the security team
- Also have a following rotation of QA/verification of fixes?
- The business case for additional (full time?) headcount can be generated by measuring the improvement from the 1 week cycle
- Create a list of backport candidates in a server PPA which users can test and nominate for backporting
Blueprint: server-o-syslog-information
Action Items
******** F I X M E ********
Blueprint: server-o-tomcat7-packaging
Action Items
By Alpha 2:
- [james-page] work with debian upstream on packaging
- [james-page] address upstart/init co-existence in debian package
- [james-page] Investigate OSGi wrapping in Jars
Blueprint: server-o-trusted-cloud
Action Items
- [action] get the virtual TPM bits packaged
[action] get the EVM tools into Ubuntu http://meego.gitorious.org/meego-platform-security/evm-utils
Blueprint: server-o-virt-tools
Action Items
cloud-utils -> non-native package (so it can be packaged in debian)
- euca2ools update
- libguestfs
- koan/virutal machine builder
- convirt update would support kvmt
- Convirt is currently in universe, Canonical OEM Services requesting an MIR
- improvements on bridging setup for guests
- [eric williams] file bug that would do bridge setup and use it for system bus.
- documentation or utility to easily set that up.
- subscribe soren and smoser
- soren wants to have p2v and v2p
- potentially a virt-manager update would get ui for snapshotting
- document or possibly change some defaults at some places for qed format (qemu-img create -f qed)
Blueprint: server-o-windows-paravirtual-drivers
Action Items
- Build the latest drivers from the latest source (ahs3)
Find out if we want to sign, WHQL, support (possibly code updates), Logo, assign build/test responsibilities for driver updates/LTS releases (MarkBaker)
- Get input on release(s) to certify from hallyn (mahmoh): target latest LTS + latest Natty
- Test the drivers as possible - 1) 2008 R2/SCSI on LTS, 2) Natty? (mahmoh)
- Compare driver performance to existing WHQLed performance (wishlist)
Blueprint: server-o-xen-host
Action Items
- canonical-kernel-team: enable Xen dom0 support in kernel.
- canonical-kernel-team: determine if Xen dom0 support requires a new flavour
- Sync Xen 4.1, with Debian packaging
- In oneiric, hypervisor and tools probably in Universe.
- pygrub to support oneiric grub2 config
- stretch goal: pvgrub2: configuration would be a static pvgrub2 image which is configured to chain load a guest pvgrub2 from a fixed location in guest filesystem; guest pvgrub2 would find and boot guest kernel
- requires grub2 xenstore client (device enumeration), vbd/blkif module, appropriate mkimage/boot support, may possibly become a new grub2 platform - cjwatson intends to work on this
"Just Do It" Blueprints
These are either previously approved, but deferred blueprints, or work that requires little to no discussion (i.e. mailing list/irc exchange is enough)
Other Team Blueprints
These are blueprints for Server, but delivered by other teams.