AutomatedServerInstalls

Differences between revisions 6 and 40 (spanning 34 versions)
Revision 6 as of 2019-07-12 03:46:30
Size: 6850
Editor: mwhudson
Comment:
Revision 40 as of 2019-08-05 02:22:10
Size: 6071
Editor: mwhudson
Comment:
Deletions are marked like this. Additions are marked like this.
Line 7: Line 7:
Please direct feedback on this proposal to $TBD. Please direct feedback on this proposal to [[https://community.ubuntu.com/t/please-review-design-for-automated-server-installs/11923|community.ubuntu.com]] or [[https://lists.ubuntu.com/mailman/listinfo/ubuntu-server|the ubuntu-server mailing list]].
Line 13: Line 13:
The server installer for 20.04 supports a new mode of operation: automated (or unattended, or hands-off) installation, autoinstallation for short. The server installer for 20.04 supports a new mode of operation: automated installation, autoinstallation for short. You might also know this feature as unattended or handsoff or preseeded installation.
Line 15: Line 15:
This lets you answer all those configuration questions ahead of time in an ''autoinstall file'' and lets the installation process run without any interaction. Autoinstallation lets you answer all those configuration questions ahead of time in an ''autoinstall file'' and lets the installation process run without any interaction.
Line 17: Line 17:
= Differences from preseeding = = Differences from debian-installer preseeding =
Line 24: Line 24:
 * when the answer to a question is not present in a preseed, d-i stops and asks the user for input. autoinstalls are not like this: if there is any autoinstall config at all, the installer takes the default for any unanswered question  * when the answer to a question is not present in a preseed, d-i stops and asks the user for input. autoinstalls are not like this: by default, if there is any autoinstall config at all, the installer takes the default for any unanswered question (and fails if there is no default). You can designate particular sections in the config as "interactive", which means the installer will still stop and ask about those.
Line 30: Line 30:
 * As /autoinstall.cfg in the initrd
 * As /autoinstall.cfg on the install media (in any partition!)
 * As `/autoinstall.yaml` in the initrd
 * As `/autoinstall.yaml` on the install media
* As `/autoinstall.yaml` on a filesystem with label "autoinstall"
Line 33: Line 34:

And maybe these ways if people think they would be useful:

 * As a b64encoded gzipped blob on the kernel command line
 * Given as a URL via DHCP
Line 41: Line 37:
When any system is installed using the server installer, an autoinstall file for repeating the install is created at {{{/var/log/installer/autoinstall.cfg}}}. When any system is installed using the server installer, an autoinstall file for repeating the install is created at {{{/var/log/installer/autoinstall.yaml}}}.
Line 49: Line 45:
$ autoinstall-editor --create $ autoinstall-editor --create > my-autoinstall.yaml
Line 51: Line 47:
$ autoinstall-editor autoinstall.cfg $ autoinstall-editor my-autoinstall.yaml
Line 56: Line 52:
The autoinstall file is YAML. Here is an example file that shows off most features: The autoinstall file is YAML and has [[/ConfigReference|full documentation]].
Line 58: Line 54:
{{{#!yaml A minimal config (for a single disk system) is:

{{{
Line 60: Line 58:
early_commands:
    - ping -c1 198.162.1.1
locale: en_US
keyboard:
    layout: en
    variant: uk
network:
    version: 2
    network:
        eth0:
            dhcp4: yes
proxy: http://squid.internal:3128/
mirror: http://repo.internal/
filesystem:
    recipe:
        name: lvm
Line 79: Line 61:
ssh:
    authorized_keys:
      - $key
    allow_pw: no
snaps:
    - go/stable
debconf_selections: |
    bind9 bind9/run-resolvconf boolean false
packages:
    - libreoffice
    - dns-server^
late_commands:
    - rm -rf /etc/init.d
Line 94: Line 63:
Many keys and values correspond straightforwardly to questions the installer asks (e.g. keyboard selection). There are some new options though: Here is an example file that shows off most features:
Line 96: Line 65:
 * `early_commands`: shell commands run after the installer has started, but before anything else (in particular, before scanning the system for block devices)
 * `debconf_selections` & `packages`: packages and configuration for them to be installed after installation has completed
 * `late_commands`: shell commands to run after the install has completed and any updates and packages installed, just before the system reboots
[[/ConfigReference#version|version]]`: 1`<<BR>>
[[/ConfigReference#reporting|reporting]]:<<BR>>
` hook:`<<BR>>
` type: webhook`<<BR>>
` endpoint: http://example.com/endpoint/path`<<BR>>
[[/ConfigReference#early_commands|early_commands]]`:`<<BR>>
` - ping -c1 198.162.1.1`<<BR>>
[[/ConfigReference#locale|locale]]`: en_US`<<BR>>
[[/ConfigReference#keyboard|keyboard]]`:`<<BR>>
` layout: en`<<BR>>
` variant: uk`<<BR>>
[[/ConfigReference#network|network]]`:`<<BR>>
` version: 2`<<BR>>
` network:`<<BR>>
` ethernets:`<<BR>>
` enp0s25:`<<BR>>
` dhcp4: yes`<<BR>>
` enp3s0:`<<BR>>
` enp4s0:`<<BR>>
` bonds:`<<BR>>
` bond0:`<<BR>>
` dhcp4: yes`<<BR>>
` interfaces:`<<BR>>
` - enp3s0`<<BR>>
` - enp4s0`<<BR>>
` parameters:`<<BR>>
` mode: active-backup`<<BR>>
` primary: enp3s0`<<BR>>
[[/ConfigReference#proxy|proxy]]`: http://squid.internal:3128/`<<BR>>
[[/ConfigReference#mirror|mirror]]`: http://repo.internal/`<<BR>>
[[/ConfigReference#filesystem|filesystem]]`:`<<BR>>
` layout:`<<BR>>
` name: lvm`<<BR>>
[[/ConfigReference#identity|identity]]`:`<<BR>>
` username: mwhudson`<<BR>>
` password: $crypted_pass`<<BR>>
[[/ConfigReference#ssh|ssh]]`:`<<BR>>
` install_server: yes`<<BR>>
` authorized_keys:`<<BR>>
` - $key`<<BR>>
` allow_pw: no`<<BR>>
[[/ConfigReference#snaps|snaps]]`:`<<BR>>
` - go/stable`<<BR>>
[[/ConfigReference#debconf_selections|debconf_selections]]`: |`<<BR>>
` bind9 bind9/run-resolvconf boolean false`<<BR>>
[[/ConfigReference#packages|packages]]`:`<<BR>>
` - libreoffice`<<BR>>
` - dns-server^`<<BR>>
[[/ConfigReference#user-data|user-data]]`:`<<BR>>
` disable_root: false`<<BR>>
[[/ConfigReference#late_commands|late_commands]]`:`<<BR>>
` - sed -ie 's/GRUB_TIMEOUT=.*/GRUB_TIMEOUT=30/' /target/etc/default/grub`<<BR>>
[[/ConfigReference#error_commands|error_commands]]`:`<<BR>>
` - tar c /var/log/installer | nc 192.168.0.1 1000`
Line 100: Line 120:
XXX should support reporting status to some endpoint? Many keys and values correspond straightforwardly to questions the installer asks (e.g. keyboard selection). See the reference for details of those that do not.
Line 102: Line 122:
== Filesystem configuration == = Possible future directions =
Line 104: Line 124:
Filesystem configuration is a complex topic and the description of the desired configuration in the autoinstall file can necessarily also be complex. The installer does support "recipes", simple ways of expressing common configurations. There are other places we could put the autoinstall config:
Line 106: Line 126:
=== Supported recipes ===  * As a b64encoded gzipped blob on the kernel command line
 * Given as a URL via DHCP
Line 108: Line 129:
The two supported recipes at the time of writing are "lvm" and "disk". Each supports a match spec (see below) to define which disk they apply to (if omitted, a disk is chosen arbitrarily). We might want to extend the 'match specs' for disks to cover other ways of selecting disks.
Line 110: Line 131:
{{{
filesystem:
  recipe:
    name: lvm
    match:
      serial: CT*
filesystem:
  recipe:
    name: disk
}}}
We might want to add some way of customizing the apt sources (adding a PPA, etc). Possibly just by accepting more of the curtin syntax for this.
Line 121: Line 133:
Supplying no filesystem config is equivalent to choosing the lvm recipe with no disk match spec. There are other things we could do by default in a system with multiple disks (create an LVM VG incorporating all of them, just install on one disk picked at random, install on the largest disk, etc etc).
Line 123: Line 135:
=== action-based config ===

For full flexibility, the installer allows filesystem configuration to be done using a syntax which is a superset of that supported by curtin, described at https://curtin.readthedocs.io/en/latest/topics/storage.html.

The extensions to the curtin syntax are mostly around disk selection. Curtin supported identifying disks by serial (e.g. `Crucial_CT512MX100SSD1_14250C57FECE`) or by path (e.g. `/dev/sdc`) and the server installer supports this as well. The installer additionally supports a ''match spec'' on a disk action that supports more flexible matching.

The actions in the filesystem config are processed in the order they are in the autoinstall file. Any disk action is assigned a matching disk -- chosen arbitrarily from the set of unassigned disks if there is more than one, and causing the installation to fail if there is no unassigned matching disk.

A match spec supports the following keys:

 * `model: foo`: matches a disk where ID_VENDOR=foo in udev, supporting globbing
 * `serial: foo`: matches a disk where ID_SERIAL=foo in udev, supporting globbing (the globbing support distinguishes this from specifying serial: foo directly in the disk action)
 * `ssd: yes|no`: matches a disk that is or is not an SSD (vs a rotating drive)
 * `size: largest`: take the largest disk rather than an arbitrary one if there are multiple matches
 * Anything else?

So for example, to match an arbitrary disk it is simply:

{{{
 - type: disk
   id: disk0
}}}

To match the largest ssd:

{{{
 - type: disk
   id: big-fast-disk
   match:
     ssd: yes
     size: largest
}}}

To match a Seagate drive:

{{{
 - type: disk
   id: data-disk
   match:
     model: Seagate
}}}

``autoinstall-editor`` supports creating and editing configs with arbitrary disk match specs.

Another extension to curtin syntax is how the size of partitions or LVM logical volumes are specified. Curtin just takes a size for this, but the server installer allows sizes to be specified as percentages of the containing device, or using min/priority/max as in d-i: (XXX I don't actually understand how min/priority/max works in d-i).

{{{
 - type: partition
   id: data-partition
   device: data-disk
   size: 75%
 - type: partition
   id: boot-partition
   device: boot-disk
   size:
     min: 512M
     priority: 600M
     max: 2G
}}}
We could support kickstart compatibility as in kickseed.

This document is entirely a description of something that does not yet exist

Automated Server Installs for 20.04 (and maybe 18.04.4?)

This document is entirely a description of something that does not yet exist

Please direct feedback on this proposal to community.ubuntu.com or the ubuntu-server mailing list.

Introduction

The server installer for 20.04 supports a new mode of operation: automated installation, autoinstallation for short. You might also know this feature as unattended or handsoff or preseeded installation.

Autoinstallation lets you answer all those configuration questions ahead of time in an autoinstall file and lets the installation process run without any interaction.

Differences from debian-installer preseeding

preseeds are the way to automate an installer based on debian-installer (aka d-i).

autoinstalls for the new server installer differ from preseeds in the following main ways:

  • the file format is completely different (yaml vs debconf-set-selections format)
  • when the answer to a question is not present in a preseed, d-i stops and asks the user for input. autoinstalls are not like this: by default, if there is any autoinstall config at all, the installer takes the default for any unanswered question (and fails if there is no default). You can designate particular sections in the config as "interactive", which means the installer will still stop and ask about those.

Providing the autoinstall file

The autoinstall file can be provided in the following ways:

  • As /autoinstall.yaml in the initrd

  • As /autoinstall.yaml on the install media

  • As /autoinstall.yaml on a filesystem with label "autoinstall"

  • Via a http or https (or maybe tftp) URL on the kernel command line

Creating an autoinstall file

When any system is installed using the server installer, an autoinstall file for repeating the install is created at /var/log/installer/autoinstall.yaml.

Alternatively there is a snap, autoinstall-editor, that can be used to either edit or create from scratch an autoinstall file (it is actually mostly the same code as that that runs the installation in interactive mode).

# start editing new config file
$ autoinstall-editor
# dump out to stdout a complete autoinstall config file with default answers everywhere
$ autoinstall-editor --create > my-autoinstall.yaml
# edit existing autoinstall file
$ autoinstall-editor my-autoinstall.yaml

The format of an autoinstall file

The autoinstall file is YAML and has full documentation.

A minimal config (for a single disk system) is:

version: 1
identity:
    username: mwhudson
    password: $crypted_pass

Here is an example file that shows off most features:

version: 1
reporting:
    hook:
        type: webhook
        endpoint: http://example.com/endpoint/path
early_commands:
    - ping -c1 198.162.1.1
locale: en_US
keyboard:
    layout: en
    variant: uk
network:
    version: 2
    network:
        ethernets:
            enp0s25:
               dhcp4: yes
            enp3s0:
            enp4s0:
        bonds:
            bond0:
                dhcp4: yes
                interfaces:
                    - enp3s0
                    - enp4s0
                parameters:
            mode: active-backup
            primary: enp3s0
proxy: http://squid.internal:3128/
mirror: http://repo.internal/
filesystem:
    layout:
        name: lvm
identity:
    username: mwhudson
    password: $crypted_pass
ssh:
    install_server: yes
    authorized_keys:
      - $key
    allow_pw: no
snaps:
    - go/stable
debconf_selections: |
    bind9      bind9/run-resolvconf    boolean false
packages:
    - libreoffice
    - dns-server^
user-data:
    disable_root: false
late_commands:
    - sed -ie 's/GRUB_TIMEOUT=.*/GRUB_TIMEOUT=30/' /target/etc/default/grub
error_commands:
    - tar c /var/log/installer | nc 192.168.0.1 1000

Many keys and values correspond straightforwardly to questions the installer asks (e.g. keyboard selection). See the reference for details of those that do not.

Possible future directions

There are other places we could put the autoinstall config:

  • As a b64encoded gzipped blob on the kernel command line
  • Given as a URL via DHCP

We might want to extend the 'match specs' for disks to cover other ways of selecting disks.

We might want to add some way of customizing the apt sources (adding a PPA, etc). Possibly just by accepting more of the curtin syntax for this.

There are other things we could do by default in a system with multiple disks (create an LVM VG incorporating all of them, just install on one disk picked at random, install on the largest disk, etc etc).

We could support kickstart compatibility as in kickseed.

FoundationsTeam/AutomatedServerInstalls (last edited 2020-06-11 04:19:11 by mwhudson)