ServerFlavorSpec

  • Launchpad Entry: server-flavors

  • Created: June 8, 2008

  • Contributors: ScottKitterman

  • Packages affected: DI and FAI plus additional packages per flavor defined.

Summary

Ubuntu Server is a general-purpose product, yet most servers are deployed for one specific purpose. This spec delivers a way to encapsulate deployment knowledge, either in Ubuntu Server as shipped or at installation time, so that administrators can install a server for a specific purpose. Most servers fit within a fairly small number of well-understood server configurations such as mail gateway, print server, proxy server, etc. Currently, in all but a few cases identified in tasksel, custom setup and configuration is required to create one of these configurations. The goal in this spec is to develop a standard method for delivering a custom 'flavor' of Ubuntu Server to meet specific use cases. This must cover not only installation of sets of packages, but integration and configuration of the packages to function in a way that meets the defined use case.

The technical approach is to use Debian Installer with preseed changes for basic installation, which late in the install calls FAI (Fully Automatic Install) to install a specific set of packages and configurations defined in FAI classes. There will be pre-defined scenarios (preseeds plus FAI classes) and a scenario editor to create a configuration for a particular install. A vital part of the scenario editor is an option to upload custom configurations to help improve the flavors shipped by default. The FAI install and configuration classes can also be used on an existing base install. It's not limited to ISO image generation.

Release Note

Ubuntu Server now provides a mechanism for easily installing customized sets of packages with an appropriate integrated configuration for a specific task. The initial 'flavor' provided with the release is a mail gateway configuration designed to do spam and virus filtering and then pass mail on to an internal server for final delivery. The intent of this new feature is to simplify deployment of Ubuntu server to perform functions defined by a particular 'flavor'. Tools for users to develop their own flavors are also included to assist large rollouts of specific configurations.

Rationale

This feature is meant to solve two different problems:

1. An admin new to Linux needs to solve a particular problem. If there is a flavor defined for the needed function, the admin can quickly deploy servers to meet the need without a lot of research and manual configuration.

2. A technically proficient admin in a large entity needs to do a large rollout of many servers with the same/very similar configurations. The admin can define a custom flavor and use the provided tools to produce appropriate images for many installs. May integrate with PXE Boot later for even more scalability.

Use Cases

Bob's boss is getting tired of spam in the inbox and the load that all the spam filtering additions puts on the company's Exchange server. Bob has heard good things about Linux and Spamassassin, but has no idea where to start because he's only ever administered Windows servers. Bob looks starts to look into setting up a Linux based MX relay, but gets overwhelmed with options and many, many new things he has to learn. Then he discovers the Mail Gateway flavor. He boots an Ubuntu Live CD and then installs and runs the scenario editor. The editor and associated tools get the information needed from him and spin a custom ISO for his exact configuration. Bob burns the ISO to a CD. Bob then boots his new server box with the CD and when it finishes, his server is configured and ready to go to work.

Jane is an admin at a large hosting company. They are looking for options to populate a new data center. Jane finds the Ubuntu Server flavor approach attractive. With a reasonable level of effort, she can define a specific set of packages and configurations for the standard web, DNS, and mail servers in the new data center. Using the scenario editor she can easily define per machine specifics. She creates images on USB sticks and then boots each server with the appropriate stick. Once install completes, the box is ready for testing and integration into the data center. Jane is impressed how easy it is to roll out a lot of Ubuntu servers.

Richard works for a company that produces a network appliance based on a standard configuration of FOSS components. It's a bit of a mess trying to manage using the tool set they've been using. Richard takes the mess of different configuration scripts and tools his company has been using and then defines a custom Ubuntu Server flavor with the packages and configurations his company has been using. He then automates the rollout process for each box by scripting the scenario specifics for each customer. His boss is thrilled because production is much more automated and configuration management is a snap. Richard is a hero.

Assumptions

The late call to FAI to install and configure packages is acting as an agent of the user and so FAI can change multiple packages. These changes will be preserved on upgrade (or the user will be asked to pick).

Design

DI + FAI classes called at late install to install/configure appropriate package set.

Scenario editor to provide per box specifics.

"Flavor Description Language"

A standardized format for describing a special purpose system configuration. It specifies an overall system type (feature set) and special characteristics within that type. The definition consists of system configuration, additional packages, integration configuration (package configuration changes to support integration of multiple packages), and package specific configuration changes. All of these items can be affected depending on system type and selected characteristics.

Implementation

Define a configuration/scenario editor to allow instance specific variables to be defined.

Develop a conversion tool to take the user input for the scenario editor and produce FAI classes

Produce a modified Server Flavor CD image base with preseed adjustements to call FAI for installation and configuration

Implement a sample flavor project

Documentation

Migration

Not applicable for initial release.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release.

This need not be added or completed until the specification is nearing beta.

Outstanding Issues

A design decision needs to be made between producing a per-installation CD image based on the output of the scenario editor or having a standard flavor CD image that will pull the per-installation specifics from a remote source.

BoF agenda and discussion

Notes from UDS Intrepid discussion:

Server Package Integration

We need to have a mechanism to install many different application stacks for different server uses.

  • - We are calling these scenarios. - The system must be scalable (support many different scearios), be policy compliant, and allow installation of many different packages with unique configurations per scenario. - Mail server is the initial scenario, but there are many that can be used, so our mechanism needs to be scalable to a fairly arbitrary number of scenarios.

Need to deal with managing config files on install and upgrades

Possibly use FAI called from DI in late install

  • - Need to get the scenario space defined and mounted

FAI can be an installer, but as an installer is 'poor'. DI lacks a good understanding of classes of packages. Using DI to install and FAI to configure a particular scenario is using each tool to do what it does best.

Mail Integration

  • Mail Gateway -- Virus, Spam, Content filtering.
    • - Postfix for MTA/MSA - Amavisd-new for integration of post-SMTP mail filtering (Spamassassin/Clamav) - Dovecot for SASL

There are many other possible mail scenarios.

Scenario Editor

[ ATTENTION: the scenario editor here is vaporware. It presents the ideal world ]

We can generalize the idea of installing a server providing a service to a scenario. A scenario basically consists of the following concepts:

  • - Scenario Configuration (Variables that affect the concrete parameters of
    • the server used in that scenario)
    - List of packages in the archive used to provide the scenario - Scripts, that implement the scenario configuration to the (freshly)
    • installed system.

The scenario editor can be implemented either

  • - as stand-anlone application (think gtk2 or webapp). - integrated into the installation part.

In the first case, the scenario editor would create an installation media on either pendrive or iso image to boot from cd. Also a pure netinst image for use with cobbler or similar is imaginable.

The second variant would support interactive installs only. since it prompts the adminstrator to enter the scenario configuration.

Fai - the Fully unattended installation And configuration Infrastructure

During a fai run the following steps (or stages) are done:

  1. classes: Fai runs shell scripts to determine the classes, in which the
    • particular machine participates. Each class may define shell variables, representing the scenario configration
  2. disc_config: Fai sets up partitions, creates file systems, etc.
  3. packages: Fai non-interactively installs the set of packages that belong to
    • the determined classes
  4. scripts: here arbitrary scripts are run (mostly shell scripts, but can also
    • be compiled programs or any other scripting language like cfengine, python) depending on the classes defined. These scripts do have access to the (yet unconfigured) installed system, the set classes and shell variables set in stage 'classes'. With that information, they modify the system to provide the configured scenario.

Additionally, every stage can be customized by hooks, that run between the stages. Both pre-run and post-run semantics are availble (hackish, but still).

Using FAI as scenario deployment facility

If using the stand alone approach scenario editor, that application has to include the scenario configuration into the CD. The scenario designer would provide scripts to verify the scenario configuration is complete, definition of the set of packages to install and scripts, that turn a system using the scenario configuration into a configured system. These steps map directly to the stages FAI is using.

If using the integrated approach, the system image would query the administrator at install time. This can also be integrated into the classes stage of FAI, which are implemeted as shell scripts. The scripts would then interrupt the installtion, presenting the administrator the scenario he is about to install, query the scenario configuration and then proceed with installing the system.

in both approaches, the disc_config stage would have to be disabled, because d-i is taking care of this.


CategorySpec

ServerFlavorSpec (last edited 2008-08-06 16:24:09 by localhost)