Wednesday, April 10, 2013

Importing an HVM image from EC2

I have recently taken a new role at Sales Engineer at Eucalyptus and one time consuming issue I keep running into with clients is image management - specifically being able to import an EBS backed image from EC2 into Eucalyptus.

The merits of this endeavor are open for debate: I would much prefer hybrid users use separate images on different clouds and simply use cloud-init scripts, or a similar mechanism, to pull up the same application stack on each of them.  That being said, customers want it, so we try to make it as easy as possible for them.

Another Sales Engineer here at Eucalyptus wrote a great guide on how to do this with HVM images on EC2:

Important note: When using an EBS image in Eucalyptus, Eucalyptus assumes it to be an HVM (Hardware Virtual Machine) image.

I decided to take this a step further and automate it with a simple shell script.  You can grab it here:

Please note that this isn't an officially supported tool and hasn't been tested by anyone other than yours truly. That being said, to test it out, the following pre-requisites must be met:

* An AWS account
* Obtaining the AMI ID of the HVM image you'd like to import
* An instance store AMI in your account to be used as a scratch instance with root access (some Ubuntu images may not work as they use user ubuntu instead)
* Enough disk space in the TEMPDIR on your local machine to store the image
* An ssh key setup on AWS and downloaded to your home directory as <key-name>.priv

Once you have downloaded the script, make sure you have your AWS secret key and access key exported in your shell environment, and then run it as follows:

./fetch-hvm-image ami-XXXXXXXX key-name [ami-XXXXXXXX]

The first AMI is the HVM AMI and is required.  The second AMI argument is optional and if left unspecified, the script will use the first instance-store AMI in your account that it finds.

If the script finishes successfully, it will display the image location in your TEMPDIR.  From there you can import it manually into your Eucalyptus setup or use the new, unofficial version of eustore mentioned in my previous blog post to do it for you.

Please note that many images will require modifications before importation.  For instance, the official SuSe images have startup scripts in them specific to AWS that can cause boot problems in Euca an must be disabled.  If you run into problems, feel free to post about it here, and I will do my best to help you out.


Wednesday, March 20, 2013

eustore now supports bootable EBS images

Those of you familiar with Eucalyptus are likely already familiar with the eustore client, which is a utility included with the popular euca2ools.  It allows for the automatic fetching, uploading, and registering of instance-store based images in your Eucalyptus cloud.

It pulls its meta-data and images from but currently filters on only our most tested images which include Fedora 16/17, Centos 5/6, and OpenSuSe 12.2.  Unfortunately, it provides no support for bootable EBS images.

However, as of today, you can beta test a new version of euca2ools that does include the ability for eustore to fetch, upload, and register your bootable EMS image:

yum install

The new functionality is contained within the utility eustore-install-image.  To use it, you will need a bootable EBS image available on an accessible http server.  From there, you can simply point eustore-install-image to it and watch it work:

eustore-install-image -u <> -v <image_size_in_GB> -k <eucalyptus_ssh_key>

You must have your SSH key located in your home directory and it should be named <keyname>.priv.  The utility assumes that you are passing it an unpacked EBS image bootable from /dev/sda1.

Please note that this has only been tested internally on a limited number of images and all feedback is not only welcomed but encouraged.  Enjoy!

Tuesday, February 5, 2013

Some of you may have heard that Eucalyptus no longer supports Debian or Ubuntu.
While it is true that given our current customer base and target customer base
that we have decided to support only CentOS and RHEL from a contractual
standpoint with our customers, it does not mean we have abandoned our
community efforts behind Debian and Ubuntu.

In fact, the opposite is true.  There was a long period of time where Eucalyptus
was not available in Debian and/or Ubuntu or only very old versions were
available.  Today, Eucalyptus 3.1 is available in Debian Unstable (we narrowly
missed the deadline to be included in Wheezy) and in Ubuntu Precise via the
Canonical Partner Repository.  Thanks goes out to James Page and others who made
that happen in Ubuntu.

The reason Eucalyptus 3.2 has not been made available in Debian is simply
because changes made to Eucalyptus require a new version of the Google Guava
libraries.  The Debian Java Team has already packaged the needed version and
uploaded it to Debian Experimental.  As soon as they deem it worthy for inclusion
in Unstable, we will look at updating Eucalyptus there as well.

We likely won't be making  any further release via the Canonical Partner
Repository as future versions of Ubuntu will have the Eucalyptus packages in
Ubuntu's Universe Repository.

For those of you who would like to build Debian Packages for Eucalyptus
from one of our development branches - in case you need one of the new features
it provides or perhaps just because you like to live on the bleeding edge - that
too is possible.

Building your own Debian Packages

Before building your own Debian packages, you will need to have the following:

 * A basic knowledge of git an bzr
 * The packages git, bzr, dpkg-dev, devscripts, pbuilder, and aptitude installed
 * A git clone of eucalyptus from github
 * The debian directory containing the proper files for building a package

Here is an example of grabbing the Eucalyptus source code and the Debian build
files, and building the Debian packages that make up Eucalyptus:

apt-get update
apt-get insall dpkg-dev devscripts
git clone git://
bzr branch lp:~brian-thomason/+junk/eucalyptus-sid
cd eucalyptus
git checkout maint/3.2/testing
cp -a eucalyptus-sid/debian ./
dch -v 3.2.1-0 Experimental Build
sudo /usr/lib/pbuilder/pbuilder-satisfydepends
*You will need to comment out the following line:

import org.rhq.helpers.pluginAnnotations.agent.Units;

in the file:


This will be fixed in the repository itself soon.

Amd finally, build the packages:

dpkg-buildpackage -b

This will build packages from the 3.2 maintenance branch, which will soon be
frozen and released as bugfix release 3.2.1.  If you would like to build from
the next feature release branch, replace 'maint/3.2/testing' with 'testing' in
the above step.

After the Eucalyptus packages have been built, you may then install the DEB
packages using:

dpkg -i ../*.deb


These builds of Eucalyptus will require a newer version of libguava-java than is
available in Debian Unstable. Fortunately though, this package is
available in Debian Experimental and can be fetched here:

You can fetch it with wget and install it using dpkg as well.

Once installed, you may follow the install guide loaded here for instructions on
how to setup your newly installed cloud:

In the coming weeks I will expand upon this topic here by:

 * Explaining how to create an apt repo for each build you make and hosting
that static repo in S3/Walrus
 * Providing a script to automate much of the process

Best regards,
Brian Thomason
Eucalyptus Systems, Inc.