Ubuntu Wiki

• Launchpad Entry: cloud-server-n-image-rebundle

• Created: 2010-11-04

• Contributors: Scott Moser, Others

• Packages affected: cloud-utils

Summary

Release Note

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.

Rationale

User stories

Assumptions

Design

You can have subsections that better describe specific parts of the issue.

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

• data migration, if any
• redirects from old URLs to new ones, if any
• how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

 * Quality rebundle utilities / documentation. We don't have a real well documented path for rebundling an image. Ideally a solution would include booting on e in kvm, doing operations to it and then uploading it to s3 with confidence that it would work.
 * Brainstorm potential other cloud utilities

https://blueprints.launchpad.net/ubuntu/+spec/cloud-server-n-image-rebundle

A user expectation is that they can launch an EC2 or UEC instance, do
some customization tasks, and then register that slightly modified image
 as a new AMI.  Changes or customization might take the form of installing
additional packages, installing some first boot scripts, or configuring a
service.  For instance store instances, this will commonly involve copying
a live (booted) filesystem.  This is commonly known as "rebundling" an
instance.  2 tools that do this are ec2-rebundle-image and ec2-create-image.

I personally have always much preferred the user to more cleanly modify
their ubuntu image (ie, via loop mount and chroot or other mechanism) and
then bundle, upload and register the modified image.  It seems that may
be unrealistic.

We don't have a really well documented path for rebundling an image. 
Ideally a solution would include customizing under kvm or another hypervisor
and then uploading the modified image to cloud service.

Should be noted that this is not suitable for public AMI creation.  Should only
be used to make private images.


===
Rebundling goals:
 * support for rebundling:
   * kvm instance
   * instance store image
   * ebs root instance
 * provide tools or document how to cleanly (loop mount)
   take Ubuntu images and modify them with a script
 * create a cleanup utility to remove secrets from the image
 * create a utility that takes an EMI ID and uploads it to EC2 in one step, 
   possibly not limited to Ubuntu
 * utility to copy EBS boot AMIs from one region to the other. Involves:
   * starting EC2 instances
   * creating volumes and snapshot
   * rsync file system
   * registering snapshot as AMI
   * deleting volumes
   * terminating instances
   * managing ssh keys for rsync between instances in different regions
   * handling failures (where some involve ongoing costs to the user)

Rebundling Questions:
 * what about cleaning ?  How should that be handled.
  * /var/log/..
  * /root/.ssh/authorized_keys
  * ~/.bash_history
  * ~/.mysql_history

Other tools:
  * kirkland: It's really a pain to have to go to [1] every time I want to
    launch an instance. I think you have some xc2-* helpers to simplify this,
    don't you Scott?
    
 * uec-run-instances take public key, not just username on launchpad for inserting, and insert via userdata
 
Maverick leftovers:
(2)send mail requesting feedback on rebundle design: TODO
(2)rebundle image (ec2-ubuntu-base) into cloud-utils in PPA: TODO
(2)rebundle support for re-bundling an image to EBS: TODO
(2)rebundle support for re-bundling an image to instance store: TODO
(2)rebundle support for "let the user do some things by hand": TODO
(3)euca2ools/ec2-api-tools wrapper into cloud-utils: TODO
(3)wrapper name based launching: TODO
(3)published image to tarfile : TODO
(3)uec-cc-setup into cloud-utils: TODO
(3)util for kill all instances: TODO
(3)util for delete instance (by ami or name): TODO
(3)delete all in a bucket (unregistering): TODO

NARWHAL NARWHAL

smoser's notes
* cleaning tool
 * needs to undo what uncloud-init does.
 * need config file or options for doing specific things
   * remove history files
   * remove .ssh/authorized_keys
   * remove /var/log/...
   * remove /var/lib/cloud/
* fix euca-bundle-vol filesystem copy (LP: #??????)
* fix ec2-bundle-vol filesystem copy
* fix euca-bundle-vol label copy (LP: #??????)
* fix ec2-bundle-vol label copy
* fix euca-bundle-vol / ec2-bundle-vol bugs
* Documentation needs to:
  * document the cleaning utility above
  * suggest euca-bundle-vol should be used for instance store images
  * suggest CreateImage should be used for ebs
  * warns users that it is dangerous to rebundle and should not be
    used for public image creation. clean image creation
    is better (ie, something like ec2-ubuntu-base)
  * covers publishing an image from local mods under kvm
* Other useful utilities:
  * migrate region ebs root utility
  * tool that takes an EMI ID and uploads it to EC2 in one step, 
    possibly not limited to Ubuntu
  * uec-run-instances take public key, for inserting, and insert via userdata
  * (3)published image to tarfile : TODO
  * (3)uec-cc-setup into cloud-utils: TODO
  * (3)util for kill all instances: TODO
  * (3)util for delete instance (by ami or name): TODO
  * (3)delete all in a bucket (unregistering): TODO
  * torsten: clean cloud utility

CategorySpec