Ubuntu Wiki

This is work in progress

This document is intended to be a quick reference for someone trying to bring up an orchestra server in order to have other machines (physical or VMs) automatically installed.

+-------------------------------------------------------------------------------+
|                                    Orchestra Host                             |
| +-------------+  +------------+  +-------------+  +-------------+  +---------+|
| | DHCP Server*|  | DNS Server*|  | TFTP Server |  | Squid Proxy |  | Cobbler ||
| +-------------+  +------------+  +-------------+  +-------------+  +---------+|
+------------------------------+------------------------------------------------+
                               |
            +------------------+------+---...
            |                         |
+-----------+------------+  +---------+-------+
|     KVM/Xen Host       |  | Physical Target |  ...
++------+  +------+      |  +-----------------+
|| VM 1 |  | VM 2 |  ... |
|+------+  +------+      |
+------------------------+

Software Installation

The orchestra host started from a 11.10 (Oneiric) server installation. In my network, I already have a DHCP and DNS service running, so I wanted those services to remain outside the orchestra host.

#> sudo apt-get install ubuntu-orchestra-server

This will install all the required software. During installation a dialogue will pop up and ask about having DHCP and DNS services running locally or not. So I said no there. One also has to set the administrator password for cobbler (and another one which I cannot remember right now).

Configuration

Directory permissions for ''/var/log/orchestra/rsyslog''

This is a known bug at least for 11.10 (Oneiric), it may be gone in later releases.

Make sure the directory is owned by user and group syslog, otherwise there will be a lot of error messages in the local syslog.

#> sudo chown syslog.syslog /var/log/orchestra/rsyslog
#> ls -la /var/log/orchestra/rsyslog
total 12
drwxr-xr-x  3 syslog syslog 4096 2012-01-17 15:44 .
drwxr-xr-x 18 root   root   4096 2012-01-19 07:43 ..
drwxr-xr-x  3 syslog syslog 4096 2012-01-18 14:31 rsyslog

Enable PXE boot on the DHCP server

Only needed, when the DHCP service is running on a different host.

The DHCP server has to set the next-server and boot file name. My DHCP server is still running 10.04 (Lucid) which uses the dhcp3-server package. So the configuration there is in /etc/dhcp3/dhcpd.conf. On later releases this might be /etc/dhcp/dhcpd.conf.

• subnet xxx.xxx.xxx.xxx netmask yyy.yyy.yyy.yyy {
    ...
    filename "pxelinux.0";
    next-server <ip of the orchestra host>;
  }

Make use of existing package mirror

I am already running apt-cacher-ng (incidentally on the same machine as orchestra), so it seems a waste not to use that. Others even may have full mirrors. This can be change in the orchestry_proxy snippet via the web interface (Configuration->Snippets->orchestra_proxy). Since my proxy runs on the same machine I only had to change the port:

• # Point apt to orchestra-provisioning-server's squid
  d-i     mirror/country string manual
  d-i     mirror/http/hostname string archive.ubuntu.com
  d-i     mirror/http/directory string /ubuntu
  d-i     mirror/http/proxy string http://@@server@@:3142/

Not sure it is also required for that change, but it will not hurt to do a sync after changing this (Actions->Sync).

Enabling PXE boot for virtualization hosts

This problem probably comes up as a result of how libvirt usually sets up guests. By default libvirt installations set up a NAT bridged subnet called default and this subnet is used when creating new virtual machines. However the TFTP service does not work over subnet boundaries. This can be solved in two ways:

1. Run a local TFTP server on the VM host and modify the virtual subnet's DHCP server to enable it.
2. Create a transparent bridge on the VM host and change the guests to use that.

Keeping the virtual machines in their own subnet probably has some advantages, too. It is probably harder to run tests pushed from a central test server and one needs to keep the local TFTP data in sync with the orchestra server. But for reference I write up both approaches:

1. Local TFTP server

   Libvirt uses dnsmasq to provide the DHCP service for the virtual subnets. That can also provide a TFTP service. The only hurdle is that the configuration can not be done through the GUI. So on the VM host, when there are no virtual machines running, run:

    #> sudo virsh
    Welcome to virsh, the virtualization interactive terminal.
    
    Type:  'help' for help with commands
           'quit' to quit
    
    virsh # connect
    virsh # net-destroy default
    virsh # net-edit default
    virsh # net-create /etc/libvirt/qemu/networks/default.xml
    virsh # quit

   When running net-edit an editor pops up with the current contents of the default network configuration. The tags to add are tftp and bootp:

     <network>
     <name>default</name>
     ...
      <ip address='192.168.122.1' netmask='255.255.255.0'>
        <tftp root='/var/lib/tftpboot' />
        <dhcp>
          <range start='192.168.122.2' end='192.168.122.254' />
          <bootp file='pxelinux.0' />
        </dhcp>
      </ip>
    </network>

   After creating the default network again, libvirt automatically starts dnsmasq to act as a TFTP server, as well as the DHCP server. The orchestra installation puts the TFTP files into /var/lib/tftpboot on the orchestra host. This needs to be synced to the VM host in some way (NFS mount, rsync, ...).

2. Transparent bridge

   This has to be set up directly in /etc/network/interfaces. The advantage is that by this all of the virtual machine share the same subnet as the real machines and can be easily reached.

    ...
    auto eth0
    iface eth0 inet manual
   
    auto br0
    iface bro inet static
      address        <vm host ip>
      netmask        x.x.x.x
      broadcast      y.y.y.y
      bridge-ports   eth0
      bridge-stp     off
      bridge-maxwait 0
      bridge-fd      0
      post-up        ip link set br0 address zz:zz:zz:zz:zz:zz

   Apparently the post-up statement is required because otherwise the bridge would use the lowest MAC address of the connected ports. I have not really checked whether this would really matter as when the bridge is brought up only eth0 is connected and one would hope the bridge MAC is not modified dynamically. So using the MAC of eth0 in the post-up statement may be completely bogus, but it at least does work.

Using the web front-end

The web front-end is running on the orchestra host under cobbler_web, so the URL would look like this:

• http://<orchestra host>/cobbler_web

The admin user name is cobbler and the password is the one which was set during installation.