CloudServerNattyCloudImages
Launchpad Entry: cloud-server-n-cloud-images
Created: 2010-11-04
Contributors: Scott Moser
Packages affected: cloud-init
Summary
Improvements to the UEC Images and associated documentation for the Natty Narwhal Cycle.
Release Note
Documentation covering the UEC images has been improved. Additionally, the images are now available in OVF format.
Rationale
UEC images provide convenient and easy to use images using Ubuntu.
User stories
- John wants to run Ubuntu on a hypervisor platform supporting consumption of OVF images.
- Jenny wants to learn more about the Ubuntu images and how she can use them.
Assumptions
Design
rescue boot
Both EBS and instance-store images can have a EBS block device attached. The design of a rescue boot would be to change the initramfs code to check for presense of a block device labelled 'RESCUE'. If there was such a block device on boot, it would be used as the root device (pivotroot).
Canonical could make rescue images available, or simply document how to
- launch instance from good image (ubuntu image) with no deletion of volume (--block-device-mapping)
- e2label /dev/sda1
- possibly insert ssh key
- stop instance
- attach volume to broken instance
- reboot broken instance
- ssh rescue instance
- touch "DONE" file
- reboot instance
touching 'DONE' above would be a way to avoid a race where you have to get off that volume, but can't detach it (ie, on reboot it would still be present, but would be ignored due to 'DONE' file). Alternatively you could just change the label back.
Implementation
Code Changes
Code changes will be primarily limited to:
- documentation
- cloud-init package
- automated-ec2-builds and ec2-publishing-scripts
Test/Demo Plan
Need to test consumption of OVF on
- vmware
- virtual box
Need to test cluster compute instances on EC2.
Unresolved issues
BoF agenda and discussion
Discussion Points: * Cluster Compute Images on EC2 During the maverick cycle announced Cluster Compute nodes. These are reasonably high powered instances for cluster computing. They utilize xen hvm for this. In order to produce this type of images we will have to * modify build process to produce another set of amis * support pv-on-hvm drivers in our kernel * possibly produce disk images rather than partition images (we may be able to create them on the fly during publishing) * OVF format By doing a couple fairly simple things, we could make the images easily comsumable by a hypervisor or system that supports OVF. The biggest immediate target would be vmware's "vApp". To accomplish this, we'll need to: * create .ova files for our images. I don't think we can get away with creating .ovf files (ie, just the manifest and re-use our .tar.gz image files) * support reading OVF ISO Transport as a DataSource in cloud-init The goal would be to support reading two OVF parameters inside the transport: user-data and meta-data (and maybe seed-from) Note: don't drop tar.gz for * Openstack support * No work supposedly needed * Quick verification this week (smoser/soren) * pre-seed installer in the maverick cycle, cloud-init gained the ability to be told where to look for its meta-data and user-data. This was in the form of a URL that would have 'user-data' and 'meta-data' appended (or a '%s' replaced). If we added a pre-seed variable to the installer that could tell cloud-init where to read its seed from, it would be possible to then do a install, and on first have cloud-init run on a "real server" Use Case: John has written a cloud-init configuration file that configures his new instances to aumatically work with xxx on UEC. He's happy with it but his manager asks him to set it up on real hardware. John would appreciate if he could reuse his work by preseeding cloud-init and adding some url on the kernel line. (talking about this with smoser, it seems that this is already possible and just need to be documented) * Better landing page for UEC Images multiple times I've wanted to point someone at our images, and the best link I can come up with is http://uec-images.ubuntu.com , which is a apache directory list. I'd like to have a general landing page for UEC images that could provide documentation, and (ideally dynamically) a list of AMIs and "latest" links for supported releases * list of AMI * how to run with kvm * general documentation about image usage: cloud-init documentation, cloud-init samples * ovf support: how to run on vmware, virtualbox. * how to rebundle images. * Reuse Cloud portal page (if cloud images are prominently exposed there) * grub fallback kernels The images are built with a kernel and a grub config that will boot. A subsequent udpate or user modification could render that grub config incorrect. It would be nice to keep a fallback stanza there with the original kernel and randisk, and ideally, on failed boot and reboot reboot into that. We don't do that in any of our other releases. Basically, I'd like to have a /boot/dist-kernel/vmlinuz and /boot/dist-kernel/initrd that were hard links to the original, and a stanza in menu.lst or grub.cfg that maintained those. Desktop already provide such a feature (to be confirmed). Use the same code for cloud images (and generic server as well). * ssh server / some console on "failed to mount root" on EC2/UEC (assuming a ramdisk is loaded) * in cases of fsck failed * or "can't mount root" (this might not be common, though) * dropbear. * Zebedee * Who: - scott, the EC2 AMI developer. - locked out users. * initrd for cloud images would have a built-in ssh server that can be started upon configuration. * logging: boot an image and sends logs to a remote syslog server asap. - cloud-config logging: output on EC2 console. - general logging server option: have a remote_syslog option in syslog and have cloud-init configure the instance to send system logs to the remote server. - user scripts output should go in a specific log file: /var/log/user_data.log * Image root fs size: - move to 10G in natty to support amazon free options. - ask amazon to support 15G so that lucid/maverick don't need to be respun. - cloud-init: provide resize option for root fs. -- [1] http://uec-images.ubuntu.com/ Proceedings: * Scott will document how to seed cloud-init from kernel command line * Scott and John will get ubuntu cluster compute types to EC2 during Natty Cycle * image root fs size for natty resized to 10G
smoser's notes: * blog entry on how to resize 15 -> 10 * Cluster Compute Images on EC2 * modify build process to produce another set of amis * support pv-on-hvm drivers in our kernel * should not need to publish intermediate files (full disk images) * OVF format * create .ova or ovf files for our images, make available on uec-images. * support reading OVF ISO Transport as a DataSource in cloud-init support reading two or three fields in metadata (user-data meta-data) * support vApp (installation via vmware) * support installation via virtual box * verify 10.10 Openstack guest support * further documentation * Better landing page for UEC Images * grub fallback kernels * Desktop already provide such a feature (to be confirmed). * ssh server / some console on "failed to mount root" on EC2/UEC * rescue root snapshots, available in ec2, available for download for uec * initrd looks for a volume labeled UEC-rescue. if attached and *not* containing file "/finished", will boot it * with grub2 (UEC), more of that can be done in grub (in original loader which would make it more resilient to root mount failure) * move to 10G in natty to support amazon free options. * cloud-init resize2fs of / by default
CloudServerNattyCloudImages (last edited 2010-11-16 21:38:37 by d14-69-66-169)