Creating a custom EC2 AMI from a qcow2 image file with Python

Question

I'm writing a service that needs to register custom AMIs in each EC2 region based on a qcow2 image file.

I have been exploring the apache-libcloud and boto libraries, but it seems the AMI registration functions are built to create an AMI based on a running instance, and I want to base the AMI on my qcow2 image file.

If there isn't an easy solution to this problem, I'll take a complex one. If for some reason this is impossible with a qcow2 image file, I also have access to the RAW image files.

Answer 1

I've succeeded in doing this programmatically. My solution uses raw image files, since they are the ones that can be directly written to a disk. If you need to convert from qcow2 image files, you can do it manually with qemu-img, or see a simple Python implementation of the conversion.

An outline of my process for AMI registration based on a raw image file:

1. Select an AMI and corresponding AKI to use as a "utility instance". It doesn't have to be the same operating system as the image you're attempting to register. If the AMI has requiretty enabled in /etc/sudoers, you need to make sure that you request a pseudo-terminal when attempting to SSH into the node, such as with Paramiko's Channel.get_pty() method.

2. Spin up a utility instance based on the AMI and AKI selected. It must be EBS optimized (m1.large size instances work well with EBS) and should have a secondary EBS volume attached that is large enough for the entire uncompressed image you want to register. I use /dev/sdb for this device name.

3. Once the utility instance is accessible via SSH, have it write the raw image file to the secondary volume. Personally, I pull a .raw.xz file from the Internet that is the image I want to write, so my utility command is sudo sh -c 'curl RAW_XZ_URL | xzcat > /dev/xvdb. Note that in all of my experience, /dev/sdX devices are accessed as /dev/xvdX on the actual instance, but this might not be the case everywhere.

4. Once the utility command completes, you can destroy the utility node, assuming that you've made your /dev/sdb volume not delete upon node termination. If you haven't, just stop the node. If executing the utility command programmatically, you can use Paramiko's Channel.recv_exit_status() method to wait until the command completes, and then check for a 0 exit status indicating success.

5. Once the utility instance is no longer running, take a snapshot of the /dev/sdb volume.

6. Once the snapshot completes, you can register it as an AMI. Make sure to use the same AKI that you've been using this whole time, as well as the proper root device name (I use full disk images, so my root device name is /dev/sda rather than /dev/sda1). Amazon suggests you use hd0 pv-grub AKIs nowadays, not hd00.

One way this can all be accomplished is through the apache-libcloud and paramiko Python libraries, both pip-installable. A good example is the Fedimg library, which implements this exact method in order to automatically register new AMIs in all EC2 regions as Fedora cloud image builds complete.

When actually implementing this process, there is quite a bit of timing, exception-handling, and other "gotchas" involved. This is simply an outline of the steps one must take to resolve the challenge via my method.