Development

Contents

  1. Preparation! You will need
  2. Detailed Deployment Instructions
    1. Ask Yourself, Is This Really Worth It?
    2. Install Mojo & Codetree
    3. Bootstrap Juju
    4. Setup Mojo
    5. Setup Creds
    6. Ok, Deploy!
    7. Fudge DNS
    8. Use Your Code
  3. Candidate bugs

Preparation! You will need

  1. Canonistack credentials from Enigma

  2. Permission to branch lp:canonical-mojo-specs (ask #webops for this)

  3. You must create at least 2 PPAs owned by you, named stable-phone-overlay and landing-000. Feel free to create more landing-NNN if you need to do a lot of testing.

Detailed Deployment Instructions

Ask Yourself, Is This Really Worth It?

The following instructions are a colossal pain, and there are many (many, many, many, many) cases in which it's completely irrelevant to ever do this. So ask yourself if you really need your own deployment:

  1. Are you just editing some web/json bits of lp:bileto? Bileto can be run locally without a deployment, just run run.py.

  2. Are you making some rather simple changes? In a lot of cases the unit tests are so good that you can just make your change, write a new test that ensures your change is good, and then go straight to testing in the staging instance in wendigo, which is always deployed and never requires this outrageous deployment hassle.
  3. Unless you're doing a sprint and a lot of people are vying for access to the staging instance, there's really no need for you to endure the pain that follows.

Install Mojo & Codetree

So, mojo and codetree aren't packaged for xenial as far as I can tell, so I'll be giving instructions on how to install them from source. In theory it's possible to do this from inside a xenial chroot, but just be aware that mojo depends on deploying an lxc (even if you're deploying to openstack), and lxc doesn't play nice inside of a chroot for whatever stupid reason (I spent days banging my head against this without success). Fun times!

Do this on xenial, as many packages are not available in trusty. 

   1 sudo apt-get install -y python-dev python-jinja2 python-novaclient python-swiftclient python-stevedore python-oslo.config python-debtcollector python-oslo.i18n python-oslo.serialization python-cinderclient python-concurrent.futures python-setuptools juju-deployer juju  lxc debootstrap
   2 sudo apt-get install -y --no-install-recommends bzr
   3 for project in codetree mojo; do 
   4     bzr branch lp:$project
   5     cd $project
   6     sudo python setup.py install
   7     cd ..
   8 done
   9 bzr branch lp:canonical-mojo-specs

Bootstrap Juju

   1 unzip ~/Downloads/canonistack-credentials.zip
   2 . novarc
   3 nova list
   4 swift list
   5 juju init

Now you need to decide whether you want to roll out into Canonistack (or a different Openstack cloud) or juju-local.

  • For Openstack, do 
       1 juju switch openstack
   2 juju bootstrap
   3 juju destroy-environment openstack --force  # when the bootstrap fails
   4 juju bootstrap
    Bootstrapping takes, like, an hour, so go get some coffee at this point. Also, this doesn't always succeed on the first try, so try bootstrapping a few times before asking for help.

  • For juju-local, you need to disable the basenode charm from your lp:canonical-mojo-spec branch, as that does not work in juju-local. patch this out, make sure to bzr commit this (it will be ignored otherwise), and run 

       1 sudo apt-get install -y juju-local
   2 echo "lxc.aa_profile = unconfined" | sudo tee /usr/share/lxc/config/common.conf.d/unconfined.conf
   3 juju switch local
   4 juju bootstrap
    Bootstrapping should only take a few minutes.

Setup Mojo

   1 sudo addgroup mojo
   2 sudo adduser $USER mojo
   3 newgrp mojo  # or log out and back in
   4 cat > mojorc <<EOF
   5 export MOJO_PUBLISHER_USERNAME=$OS_USERNAME
   6 export MOJO_STAGE=ue/mojo-ue-ci-train/devel/
   7 export MOJO_DOWNLOADER_AUTH_URL=$OS_AUTH_URL
   8 export MOJO_WORKSPACE=devel
   9 export MOJO_PUBLISHER_PASSWORD=$OS_PASSWORD
  10 export MOJO_PROJECT=mojo-stg-ue-ci-train
  11 export MOJO_PUBLISHER_TENANT_NAME=$OS_TENANT_NAME
  12 export MOJO_SERIES=trusty
  13 EOF
  14 . mojorc
  15 sudo install -o root -g root -m 0440 ./mojo/contrib/99-mojo-sudoers /etc/sudoers.d
  16 mojo project-new --series $MOJO_SERIES $MOJO_PROJECT
  17 sudo chown -R $USER:mojo /srv/mojo/$MOJO_PROJECT/$MOJO_SERIES/ROOTFS/srv/mojo/$MOJO_PROJECT/$MOJO_SERIES
  18 sudo chown $USER:mojo /srv/mojo/$MOJO_PROJECT
  19 sudo chmod 750 /srv/mojo/$MOJO_PROJECT
  20 sudo chown $USER:mojo /srv/mojo/$MOJO_PROJECT/$MOJO_SERIES
  21 sudo chmod 750 /var/lib/lxc/$MOJO_PROJECT.$MOJO_SERIES
  22 sudo chgrp mojo /var/lib/lxc/$MOJO_PROJECT.$MOJO_SERIES
  23 sudo mkdir -p /srv/mojo/LOCAL/$MOJO_PROJECT
  24 sudo chown $USER:mojo /srv/mojo/LOCAL/$MOJO_PROJECT
  25 sudo chmod 750 /srv/mojo/LOCAL/$MOJO_PROJECT
  26 sudo chgrp mojo /var/lib/lxc
  27 sudo chmod 750 /var/lib/lxc
  28 mojo workspace-new ~/canonical-mojo-specs/ $MOJO_WORKSPACE --stage $MOJO_STAGE

Setup Creds

The deployment needs the ssh & gpg keys you have registered with your launchpad account. WARNING: Only passwordless GPG keys will work if you intend to actually build MPs in your deployment. So you'll either need to create one of those and associate it with your lp account, or create a throw-away lp account for this purpose if you're a security-minded person. 

   1 LOCAL_DIR=/srv/mojo/LOCAL/$MOJO_PROJECT/$MOJO_STAGE
   2 mkdir -p $LOCAL_DIR/keys
   3 cp ~/.gnupg/pubring.gpg $LOCAL_DIR/keys/gpg.pub
   4 cp ~/.gnupg/secring.gpg $LOCAL_DIR/keys/gpg.sec
   5 cp ~/.ssh/id_rsa* $LOCAL_DIR/keys/
   6 touch $LOCAL_DIR/amqp.uri

Oh, and you need an oauth token, too! 

   1 bzr branch lp:cupstream2distro
   2 ./cupstream2distro/token_generator.py  # Follow prompts in browser
   3 cp ~/.launchpad.credentials $LOCAL_DIR/keys/launchpad.credentials

Generate self-signed SSL cert: 

   1 cd /srv/mojo/mojo-stg-ue-ci-train/trusty/devel/local/
   2 openssl req -x509 -nodes -newkey rsa:2048 -keyout requests.ci-train.ubuntu.com.key -out requests.ci-train.ubuntu.com.crt -days 365
   3 ln -s requests.ci-train.ubuntu.com.crt requests.ci-train.ubuntu.com_chain.crt

Ok, Deploy!

   1 mojo run ~/canonical-mojo-specs/ devel  # Might have to run this a couple times; may take hours
   2 juju set ci-train launchpad-user="$(bzr launchpad-login)" ppa-team="$(bzr launchpad-login)"

Fudge DNS

At this point you should be able to access just bileto by IP. 

   1 IP=$(juju status apache2 | sed -n 's#^ \+public-address: \(.\+\)$#\1#p' | uniq)
   2 firefox $IP

Due to the nature of apache2 being a reverseproxy for two different services, you'll need to add entries to /etc/hosts in order to access the jenkins unit. 

   1 echo $IP ci-train.staging.ubuntu.com | sudo tee -a /etc/hosts

Make sure to undo this when you're done or you'll never be able to access the real staging server again! Wink ;-)

Use Your Code

Once you have a working deployment you can inject your experimental branches according to the staging instructions.

Candidate bugs

This buglist is for potential fixing during Feb 2016 Bileto Sprint.

Easy (probably 20 lines of code or less):

Medium (requires significant new scripts/modules):

Hard (requires architectural shifts, probably):

citrain/Development (last edited 2016-02-03 20:27:34 by pitti)