user-data (cloud-init) script not executing on EC2

Question 1

Cloud-init does not accept plain bash scripts, just like that. It's a beast that eats YAML file that defines your instance (packages, ssh keys and other stuff).

Using MIME you can also send arbitrary shell scripts, but you have to MIME-encode them.

$ cat my-boothook.txt
#!/bin/sh
echo "Hello World!"
echo "This will run as soon as possible in the boot sequence"

$ cat my-user-script.txt
#!/usr/bin/perl
print "This is a user script (rc.local)\n"

$ cat my-include.txt
# these urls will be read pulled in if they were part of user-data
# comments are allowed.  The format is one url per line
http://www.ubuntu.com/robots.txt
http://www.w3schools.com/html/lastpage.htm

$ cat my-upstart-job.txt
description "a test upstart job"
start on stopped rc RUNLEVEL=[2345]
console output
task
script
echo "====BEGIN======="
echo "HELLO From an Upstart Job"
echo "=====END========"
end script

$ cat my-cloudconfig.txt
#cloud-config
ssh_import_id: [smoser]
apt_sources:
 - source: "ppa:smoser/ppa"

$ ls
my-boothook.txt     my-include.txt      my-user-script.txt
my-cloudconfig.txt  my-upstart-job.txt

$ write-mime-multipart --output=combined-userdata.txt \
   my-boothook.txt:text/cloud-boothook \
   my-include.txt:text/x-include-url \
   my-upstart-job.txt:text/upstart-job \
   my-user-script.txt:text/x-shellscript \
   my-cloudconfig.txt

$ ls -l combined-userdata.txt 
-rw-r--r-- 1 smoser smoser 1782 2010-07-01 16:08 combined-userdata.txt

The combined-userdata.txt is the file you want to paste there.

More info here:

https://help.ubuntu.com/community/CloudInit

Also note, this highly depends on the image you are using. But you say it is really cloud-init based image, so this applies. There are other cloud initiators which are not named cloud-init - then it could be different.

Question 2

Actually, cloud-init allows a single shell script as an input (though you may want to use a MIME archive for more complex setups).

The problem with the OP's script is that the first line is incorrect. You should use something like this:

#!/bin/sh

The reason for this is that, while cloud-init uses #! to recognize a user script, the operating system needs a complete shebang line in order to execute the script.

So what's happening in the OP's case is that cloud-init behaves correctly (i.e. it downloads and tries to run the script) but the operating system is unable to actually execute it.

See: Shebang (Unix) on Wikipedia

Question 3

This is a couple years old now, but for others benefit I had the same issue, and it turned out that cloud-init was running twice, from inside /etc/rc3.d . Deleting these files inside the folder allowed the userdata to run correctly:

lrwxrwxrwx  1 root root   22 Jun  5 02:49 S-1cloud-config -> ../init.d/cloud-config
lrwxrwxrwx  1 root root   20 Jun  5 02:49 S-1cloud-init -> ../init.d/cloud-init
lrwxrwxrwx  1 root root   26 Jun  5 02:49 S-1cloud-init-local -> ../init.d/cloud-init-local

Question 4

The problem is with cloud-init not allowing the user script to run on the next start-up.

First remove the cloud-init artifacts by executing: rm /var/lib/cloud/instances/*/sem/config_scripts_user

And then your userdata must look like this:

#!/bin/bash 
echo "hello!"

And then start you instance. It now works (tested).