= Debugging a Single install =

== Log file location ==

The default log file is located in '''~/.cloud-install/commands.log'''

== Juju failing to create lxc template clones ==

If `juju status` reports errors with lxc's about not being able to download the template or clone it for use then do the following:

{{{
$ sudo rm -rf /var/cache/lxc/cloud-trusty
$ sudo lxc-create -t ubuntu-cloud -n test
}}}

From there a Juju bootstrap will then re-use the already downloaded LXC image and proceed to deploy the services.

== Services not deploying correctly - inspect Juju status and logs ==

{{{#!highlight bash
$ sudo lxc-attach -n openstack-single-$USER
$ su ubuntu
$ export JUJU_HOME=~/.cloud-install/juju
$ juju status --format=tabular
}}}

if an individual service, e.g. 'glance' shows errors, you can inspect its log thusly:

  '''$ juju debug-log --replay -i glance/0'''

== "Top Level Container OS did not initialize properly" -- huh? ==

If your single install attempt fails with that message, check commands.log for the *last* error. It may say something like this:

{{{
159 [ERROR: 09-03 16:30:52, single.py:330] Container cloud-init finished with errors: ['(\'seed_random\', ProcessExecutionError("Unexpected error while running command.\\nCommand: [\'env\', \'pollinate\', \'-q\', \'--curl-opts\', \'-k\\
    ']\\nExit code: 1\\nReason: -\\nStdout: \'\'\\nStderr: \'\'",))']
160 [DEBUG: 09-03 16:30:52, utils.py:56] Traceback (most recent call last):
161   File "/usr/share/openstack/cloudinstall/utils.py", line 71, in run
162     super().run()
163   File "/usr/lib/python3.4/threading.py", line 868, in run
164     self._target(*self._args, '''self._kwargs)
165   File "/usr/share/openstack/cloudinstall/controllers/install/single.py", line 427, in do_install_async
166     self.do_install()
167   File "/usr/share/openstack/cloudinstall/controllers/install/single.py", line 453, in do_install
168     self.create_container_and_wait()
169   File "/usr/share/openstack/cloudinstall/controllers/install/single.py", line 224, in create_container_and_wait
170     while not self.cloud_init_finished(tries):
171   File "/usr/share/openstack/cloudinstall/controllers/install/single.py", line 331, in cloud_init_finished
172     raise Exception("Top-level container OS did not initialize "
173 Exception: Top-level container OS did not initialize correctly.
}}}

This most often means that the LXC container the installer created to house the installation does not have working network access to the outside world. There are a few reasons this might happen:

=== Proxy Issues ===

If you are behind a firewall that requires use of an http/s proxy, be sure you set that up in your install command line.

=== Container failing to reach the internet (Vivid and later) ===

This pertains to systemd-networkd found on Vivid and later

If you are running on a system using systemd and have configured it to use systemd-networkd, then you may need to set ''IPForward=yes'' in the ''[Network]'' section of the .network file corresponding to the interface that reaches the outside world. See ''man systemd.network'' for information about this option.

As an example, on this Wily system we edit '''/lib/systemd/network/80-container-host0.network'''

{{{
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Match]
Virtualization=container
Name=host0

[Network]
DHCP=yes
LinkLocalAddressing=yes

# Fix unresolvable network in LXC containers.
IPForward=yes
}}}

== Unable to lxc-attach to container on a Vivid (Ubuntu 15.04) install with LXC 1.1.2 ==

  It is a good idea to upgrade LXC on Vivid(15.04) to a newer release regardless of whether or not you run into this issue.

If you run into an error similar to:

{{{
lxc_container: cgmanager.c: lxc_cgmanager_enter: 694 call to cgmanager_move_pid_abs_sync failed: invalid request
lxc_container: cgmanager.c: cgm_attach: 1324 Failed to enter group /lxc/openstack-single-openstack/system.slice/cgproxy.service
lxc_container: attach.c: lxc_attach: 909 error communicating with child process
}}}

This can be resolved by upgrading to a newer version of LXC:

{{{#!highlight bash
$ sudo apt-add-repository ppa:ubuntu-lxc/stable
$ sudo apt-get update
$ sudo apt-get upgrade
}}}

This problem so far only exists on Vivid with the default LXC version of 1.1.2

== Accessing the Juju environment ==

The container houses the entire OpenStack deployment and is a good first step in troubleshooting an install.

In order to log into the host container we'll need to use '''lxc-attach'''

  '''$ sudo lxc-attach -n openstack-single-$USER'''

'''openstack-single-$USER''' is the default container used for installation. Within that container we can now inspect our Juju environment. 

  If for some reason the container name does not match the default simply running ''sudo lxc-ls -f'' will list out the container that was created, always starting with '''openstack-single'''.

{{{#!highlight bash
% su - ubuntu
% JUJU_HOME=~/.cloud-install/juju juju status
}}}

All juju commands need to be able to read from a customized '''JUJU_HOME''', the location that the installer uses is '''~/.cloud-install/juju'''.

From this point you can log into any of the services Juju knows about and start inspecting logs, charm configurations, etc.