ServerFlavorSpec
|
Size: 3085
Comment:
|
Size: 11528
Comment: Mention FAI can be used to modify an existing installation.
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 10: | Line 10: |
| There are a significant different number of semi-standard server configurations that Ubuntu Server generally has all the packages needed to support it. Currently, in all but a few cases identified in tasksel, it takes custom setup and configuration to get a functional server to meet specific requirements. The goal in this spec is to develop a standard method for delivering a custom 'flavor' of Ubuntu Server to meet specific use cases. This must cover not only installation of sets of packages, but integration and configuration of the packages to function in a way that meets the defined use case. | Ubuntu Server is a general-purpose product, yet most servers are deployed for one specific purpose. This spec delivers a way to encapsulate deployment knowledge, either in Ubuntu Server as shipped or at installation time, so that administrators can install a server for a specific purpose. Most servers fit within a fairly small number of well-understood server configurations such as mail gateway, print server, proxy server, etc. Currently, in all but a few cases identified in tasksel, custom setup and configuration is required to create one of these configurations. The goal in this spec is to develop a standard method for delivering a custom 'flavor' of Ubuntu Server to meet specific use cases. This must cover not only installation of sets of packages, but integration and configuration of the packages to function in a way that meets the defined use case. |
| Line 12: | Line 12: |
| The technical approach is to use a combination of Debian Installer with preseed changes for basic installation and FAI (Fully Automatic Install) called | The technical approach is to use Debian Installer with preseed changes for basic installation, which late in the install calls FAI (Fully Automatic Install) to install a specific set of packages and configurations defined in FAI classes. There will be pre-defined scenarios (preseeds plus FAI classes) and a scenario editor to create a configuration for a particular install. A vital part of the scenario editor is an option to upload custom configurations to help improve the flavors shipped by default. The FAI install and configuration classes can also be used on an existing base install. It's not limited to ISO image generation. |
| Line 16: | Line 17: |
| This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.) It is mandatory. |
Ubuntu Server now provides a mechanism for easily installing customized sets of packages with an appropriate integrated configuration for a specific task. The initial 'flavor' provided with the release is a mail gateway configuration designed to do spam and virus filtering and then pass mail on to an internal server for final delivery. The intent of this new feature is to simplify deployment of Ubuntu server to perform functions defined by a particular 'flavor'. Tools for users to develop their own flavors are also included to assist large rollouts of specific configurations. |
| Line 22: | Line 21: |
| This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. | This feature is meant to solve two different problems: 1. An admin new to Linux needs to solve a particular problem. If there is a flavor defined for the needed function, the admin can quickly deploy servers to meet the need without a lot of research and manual configuration. 2. A technically proficient admin in a large entity needs to do a large rollout of many servers with the same/very similar configurations. The admin can define a custom flavor and use the provided tools to produce appropriate images for many installs. May integrate with PXE Boot later for even more scalability. |
| Line 26: | Line 29: |
| Bob's boss is getting tired of spam in the inbox and the load that all the spam filtering additions puts on the company's Exchange server. Bob has heard good things about Linux and Spamassassin, but has no idea where to start because he's only ever administered Windows servers. Bob looks starts to look into setting up a Linux based MX relay, but gets overwhelmed with options and many, many new things he has to learn. Then he discovers the Mail Gateway flavor. He boots an Ubuntu Live CD and then installs and runs the scenario editor. The editor and associated tools get the information needed from him and spin a custom ISO for his exact configuration. Bob burns the ISO to a CD. Bob then boots his new server box with the CD and when it finishes, his server is configured and ready to go to work. Jane is an admin at a large hosting company. They are looking for options to populate a new data center. Jane finds the Ubuntu Server flavor approach attractive. With a reasonable level of effort, she can define a specific set of packages and configurations for the standard web, DNS, and mail servers in the new data center. Using the scenario editor she can easily define per machine specifics. She creates images on USB sticks and then boots each server with the appropriate stick. Once install completes, the box is ready for testing and integration into the data center. Jane is impressed how easy it is to roll out a lot of Ubuntu servers. Richard works for a company that produces a network appliance based on a standard configuration of FOSS components. It's a bit of a mess trying to manage using the tool set they've been using. Richard takes the mess of different configuration scripts and tools his company has been using and then defines a custom Ubuntu Server flavor with the packages and configurations his company has been using. He then automates the rollout process for each box by scripting the scenario specifics for each customer. His boss is thrilled because production is much more automated and configuration management is a snap. Richard is a hero. |
|
| Line 27: | Line 36: |
The late call to FAI to install and configure packages is acting as an agent of the user and so FAI can change multiple packages. These changes will be preserved on upgrade (or the user will be asked to pick). |
|
| Line 30: | Line 41: |
| You can have subsections that better describe specific parts of the issue. | DI + FAI classes called at late install to install/configure appropriate package set. Scenario editor to provide per box specifics. "Flavor Description Language" A standardized format for describing a special purpose system configuration. It specifies an overall system type (feature set) and special characteristics within that type. The definition consists of system configuration, additional packages, integration configuration (package configuration changes to support integration of multiple packages), and package specific configuration changes. All of these items can be affected depending on system type and selected characteristics. |
| Line 34: | Line 51: |
| This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: | Define a configuration/scenario editor to allow instance specific variables to be defined. |
| Line 36: | Line 53: |
| === UI Changes === | Develop a conversion tool to take the user input for the scenario editor and produce FAI classes |
| Line 38: | Line 55: |
| Should cover changes required to the UI, or specific UI that is required to implement this | Produce a modified Server Flavor CD image base with preseed adjustements to call FAI for installation and configuration |
| Line 40: | Line 57: |
| === Code Changes === | Implement a sample flavor project |
| Line 42: | Line 59: |
| Code changes should include an overview of what needs to change, and in some cases even the specific details. | Documentation |
| Line 46: | Line 63: |
| Include: * data migration, if any * redirects from old URLs to new ones, if any * how users will be pointed to the new way of doing things, if necessary. |
Not applicable for initial release. |
| Line 59: | Line 73: |
| This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. | A design decision needs to be made between producing a per-installation CD image based on the output of the scenario editor or having a standard flavor CD image that will pull the per-installation specifics from a remote source. |
| Line 63: | Line 77: |
| Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected. | Notes from UDS Intrepid discussion: |
| Line 65: | Line 79: |
| == Server Package Integration == We need to have a mechanism to install many different application stacks for different server uses. - We are calling these scenarios. - The system must be scalable (support many different scearios), be policy compliant, and allow installation of many different packages with unique configurations per scenario. - Mail server is the initial scenario, but there are many that can be used, so our mechanism needs to be scalable to a fairly arbitrary number of scenarios. Need to deal with managing config files on install and upgrades Possibly use FAI called from DI in late install - Need to get the scenario space defined and mounted FAI can be an installer, but as an installer is 'poor'. DI lacks a good understanding of classes of packages. Using DI to install and FAI to configure a particular scenario is using each tool to do what it does best. == Mail Integration == * Mail Gateway -- Virus, Spam, Content filtering. - Postfix for MTA/MSA - Amavisd-new for integration of post-SMTP mail filtering (Spamassassin/Clamav) - Dovecot for SASL There are many other possible mail scenarios. === Scenario Editor === [ ATTENTION: the scenario editor here is vaporware. It presents the ideal world ] We can generalize the idea of installing a server providing a service to a scenario. A scenario basically consists of the following concepts: - Scenario Configuration (Variables that affect the concrete parameters of the server used in that scenario) - List of packages in the archive used to provide the scenario - Scripts, that implement the scenario configuration to the (freshly) installed system. The scenario editor can be implemented either - as stand-anlone application (think gtk2 or webapp). - integrated into the installation part. In the first case, the scenario editor would create an installation media on either pendrive or iso image to boot from cd. Also a pure netinst image for use with cobbler or similar is imaginable. The second variant would support interactive installs only. since it prompts the adminstrator to enter the scenario configuration. === Fai - the Fully unattended installation And configuration Infrastructure === During a fai run the following steps (or stages) are done: 1. classes: Fai runs shell scripts to determine the classes, in which the particular machine participates. Each class may define shell variables, representing the scenario configration 1. disc_config: Fai sets up partitions, creates file systems, etc. 1. packages: Fai non-interactively installs the set of packages that belong to the determined classes 1. scripts: here arbitrary scripts are run (mostly shell scripts, but can also be compiled programs or any other scripting language like cfengine, python) depending on the classes defined. These scripts do have access to the (yet unconfigured) installed system, the set classes and shell variables set in stage 'classes'. With that information, they modify the system to provide the configured scenario. Additionally, every stage can be customized by hooks, that run between the stages. Both pre-run and post-run semantics are availble (hackish, but still). === Using FAI as scenario deployment facility === If using the stand alone approach scenario editor, that application has to include the scenario configuration into the CD. The scenario designer would provide scripts to verify the scenario configuration is complete, definition of the set of packages to install and scripts, that turn a system using the scenario configuration into a configured system. These steps map directly to the stages FAI is using. If using the integrated approach, the system image would query the administrator at install time. This can also be integrated into the classes stage of FAI, which are implemeted as shell scripts. The scripts would then interrupt the installtion, presenting the administrator the scenario he is about to install, query the scenario configuration and then proceed with installing the system. in both approaches, the disc_config stage would have to be disabled, because d-i is taking care of this. |
Launchpad Entry: server-flavors
Created: June 8, 2008
Contributors: ScottKitterman
Packages affected: DI and FAI plus additional packages per flavor defined.
Summary
Ubuntu Server is a general-purpose product, yet most servers are deployed for one specific purpose. This spec delivers a way to encapsulate deployment knowledge, either in Ubuntu Server as shipped or at installation time, so that administrators can install a server for a specific purpose. Most servers fit within a fairly small number of well-understood server configurations such as mail gateway, print server, proxy server, etc. Currently, in all but a few cases identified in tasksel, custom setup and configuration is required to create one of these configurations. The goal in this spec is to develop a standard method for delivering a custom 'flavor' of Ubuntu Server to meet specific use cases. This must cover not only installation of sets of packages, but integration and configuration of the packages to function in a way that meets the defined use case.
The technical approach is to use Debian Installer with preseed changes for basic installation, which late in the install calls FAI (Fully Automatic Install) to install a specific set of packages and configurations defined in FAI classes. There will be pre-defined scenarios (preseeds plus FAI classes) and a scenario editor to create a configuration for a particular install. A vital part of the scenario editor is an option to upload custom configurations to help improve the flavors shipped by default. The FAI install and configuration classes can also be used on an existing base install. It's not limited to ISO image generation.
Release Note
Ubuntu Server now provides a mechanism for easily installing customized sets of packages with an appropriate integrated configuration for a specific task. The initial 'flavor' provided with the release is a mail gateway configuration designed to do spam and virus filtering and then pass mail on to an internal server for final delivery. The intent of this new feature is to simplify deployment of Ubuntu server to perform functions defined by a particular 'flavor'. Tools for users to develop their own flavors are also included to assist large rollouts of specific configurations.
Rationale
This feature is meant to solve two different problems:
1. An admin new to Linux needs to solve a particular problem. If there is a flavor defined for the needed function, the admin can quickly deploy servers to meet the need without a lot of research and manual configuration.
2. A technically proficient admin in a large entity needs to do a large rollout of many servers with the same/very similar configurations. The admin can define a custom flavor and use the provided tools to produce appropriate images for many installs. May integrate with PXE Boot later for even more scalability.
Use Cases
Bob's boss is getting tired of spam in the inbox and the load that all the spam filtering additions puts on the company's Exchange server. Bob has heard good things about Linux and Spamassassin, but has no idea where to start because he's only ever administered Windows servers. Bob looks starts to look into setting up a Linux based MX relay, but gets overwhelmed with options and many, many new things he has to learn. Then he discovers the Mail Gateway flavor. He boots an Ubuntu Live CD and then installs and runs the scenario editor. The editor and associated tools get the information needed from him and spin a custom ISO for his exact configuration. Bob burns the ISO to a CD. Bob then boots his new server box with the CD and when it finishes, his server is configured and ready to go to work.
Jane is an admin at a large hosting company. They are looking for options to populate a new data center. Jane finds the Ubuntu Server flavor approach attractive. With a reasonable level of effort, she can define a specific set of packages and configurations for the standard web, DNS, and mail servers in the new data center. Using the scenario editor she can easily define per machine specifics. She creates images on USB sticks and then boots each server with the appropriate stick. Once install completes, the box is ready for testing and integration into the data center. Jane is impressed how easy it is to roll out a lot of Ubuntu servers.
Richard works for a company that produces a network appliance based on a standard configuration of FOSS components. It's a bit of a mess trying to manage using the tool set they've been using. Richard takes the mess of different configuration scripts and tools his company has been using and then defines a custom Ubuntu Server flavor with the packages and configurations his company has been using. He then automates the rollout process for each box by scripting the scenario specifics for each customer. His boss is thrilled because production is much more automated and configuration management is a snap. Richard is a hero.
Assumptions
The late call to FAI to install and configure packages is acting as an agent of the user and so FAI can change multiple packages. These changes will be preserved on upgrade (or the user will be asked to pick).
Design
DI + FAI classes called at late install to install/configure appropriate package set.
Scenario editor to provide per box specifics.
"Flavor Description Language"
A standardized format for describing a special purpose system configuration. It specifies an overall system type (feature set) and special characteristics within that type. The definition consists of system configuration, additional packages, integration configuration (package configuration changes to support integration of multiple packages), and package specific configuration changes. All of these items can be affected depending on system type and selected characteristics.
Implementation
Define a configuration/scenario editor to allow instance specific variables to be defined.
Develop a conversion tool to take the user input for the scenario editor and produce FAI classes
Produce a modified Server Flavor CD image base with preseed adjustements to call FAI for installation and configuration
Implement a sample flavor project
Documentation
Migration
Not applicable for initial release.
Test/Demo Plan
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release.
This need not be added or completed until the specification is nearing beta.
Outstanding Issues
A design decision needs to be made between producing a per-installation CD image based on the output of the scenario editor or having a standard flavor CD image that will pull the per-installation specifics from a remote source.
BoF agenda and discussion
Notes from UDS Intrepid discussion:
Server Package Integration
We need to have a mechanism to install many different application stacks for different server uses.
- - We are calling these scenarios. - The system must be scalable (support many different scearios), be policy compliant, and allow installation of many different packages with unique configurations per scenario. - Mail server is the initial scenario, but there are many that can be used, so our mechanism needs to be scalable to a fairly arbitrary number of scenarios.
Need to deal with managing config files on install and upgrades
Possibly use FAI called from DI in late install
- - Need to get the scenario space defined and mounted
FAI can be an installer, but as an installer is 'poor'. DI lacks a good understanding of classes of packages. Using DI to install and FAI to configure a particular scenario is using each tool to do what it does best.
Mail Integration
- Mail Gateway -- Virus, Spam, Content filtering.
- - Postfix for MTA/MSA - Amavisd-new for integration of post-SMTP mail filtering (Spamassassin/Clamav) - Dovecot for SASL
There are many other possible mail scenarios.
Scenario Editor
[ ATTENTION: the scenario editor here is vaporware. It presents the ideal world ]
We can generalize the idea of installing a server providing a service to a scenario. A scenario basically consists of the following concepts:
- - Scenario Configuration (Variables that affect the concrete parameters of
- the server used in that scenario)
- installed system.
The scenario editor can be implemented either
- - as stand-anlone application (think gtk2 or webapp). - integrated into the installation part.
In the first case, the scenario editor would create an installation media on either pendrive or iso image to boot from cd. Also a pure netinst image for use with cobbler or similar is imaginable.
The second variant would support interactive installs only. since it prompts the adminstrator to enter the scenario configuration.
Fai - the Fully unattended installation And configuration Infrastructure
During a fai run the following steps (or stages) are done:
- classes: Fai runs shell scripts to determine the classes, in which the
- particular machine participates. Each class may define shell variables, representing the scenario configration
- disc_config: Fai sets up partitions, creates file systems, etc.
- packages: Fai non-interactively installs the set of packages that belong to
- the determined classes
- scripts: here arbitrary scripts are run (mostly shell scripts, but can also
- be compiled programs or any other scripting language like cfengine, python) depending on the classes defined. These scripts do have access to the (yet unconfigured) installed system, the set classes and shell variables set in stage 'classes'. With that information, they modify the system to provide the configured scenario.
Additionally, every stage can be customized by hooks, that run between the stages. Both pre-run and post-run semantics are availble (hackish, but still).
Using FAI as scenario deployment facility
If using the stand alone approach scenario editor, that application has to include the scenario configuration into the CD. The scenario designer would provide scripts to verify the scenario configuration is complete, definition of the set of packages to install and scripts, that turn a system using the scenario configuration into a configured system. These steps map directly to the stages FAI is using.
If using the integrated approach, the system image would query the administrator at install time. This can also be integrated into the classes stage of FAI, which are implemeted as shell scripts. The scripts would then interrupt the installtion, presenting the administrator the scenario he is about to install, query the scenario configuration and then proceed with installing the system.
in both approaches, the disc_config stage would have to be disabled, because d-i is taking care of this.
ServerFlavorSpec (last edited 2008-08-06 16:24:09 by localhost)