How to launch an amazon ec2 instance inside VPC using Chef?

Question

This is a question primarily about Chef. When looking into controlling nodes inside Amazon VPC with Chef, I run into some difficulties, mainly that a node that does not have an external IP address is not easily reachable by chef.

I went through the basic tutorial for scenario #2 http://docs.amazonwebservices.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html#Case2_Launch_NAT

However, this this times out:

knife ec2 server create -N app-server-1 -f m1.small -i rails-quick-start.pem -r "role[base]" -G WebServerSG -S rails-quick-start -x ubuntu -s subnet-580d7e30 -y -I ami-073ae46e -Z us-east-1d

What am I doing wrong?

Answer 1

The solution was to setup a tunnel and tunnel the ssh on some port of a publicly visible computer to all the other computers in the cloud. So my load balancer serves http traffic on socket 80, is accessible via socket 22, and uses sockets 2222, 2223, 2224, ... to tunnel ssh to non-public cloud instances. On load balancer (or any public instance) run:

ncat --sh-exec "ncat PRIVATE.SUBNET.IP 22" -l 2222 &

for example:

ncat --sh-exec "ncat 10.0.1.1 22" -l 2222 &

Answer 2

In order for knife to be able to talk to the server you may need to set up a VPN. If your VPC is already connected to your local network via a VPN then it should work but if not you might want to run an OpenVPN server or something similar.

You can also set up servers in two other ways:

1. Create an EC2 instance and let it boot up. Then run knife bootstrap against it.
2. Create an EC2 instance with the proper user data and have cloud-init set it up (if you are running say ubuntu with includes cloud-init).

Answer 3

There needs to be a way to associate an Elastic IP to the instance in order to get a public IP for easy access and then do all the bootstrapping and SSH activities through the EIP.