Static content for AWS EC2 with IAM role

Question

Reading through the / many / resources on how to utilize temporary AWS credentials in a launched EC2 instance, I can't seem to get an extremely simple POC running.

Desired:

1. Launch an EC2 instance
2. SSH in
3. Pull a piece of static content from a private S3 bucket

Steps:

1. Create an IAM role
2. Spin up a new EC2 instance with the above IAM role specified; SSH in
3. Set the credentials using aws configure and the details that (successfully) populated in http://169.254.169.254/latest/meta-data/iam/security-credentials/iam-role-name
4. Attempt to use the AWS CLI directly to access the file

IAM role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::bucket-name/file.png"
    }
  ]
}

When I use the AWS CLI to access the file, this error is thrown:

A client error (Forbidden) occurred when calling the HeadObject operation: Forbidden
Completed 1 part(s) with ... file(s) remaining

Which step did I miss?

Answer 1

For future reference, the issue was in how I was calling the AWS CLI; previously I was running:

aws configure

...and supplying the details found in the auto-generated role profile.

Once I simply allowed it to find its own temporary credentials and just specified the only other required parameter manually (region):

aws s3 cp s3://bucket-name/file.png file.png --region us-east-1

...the file pulled fine. Hopefully this'll help out someone in the future!

Answer 2

Hope this might help some other Googler that lands here.

The

A client error (403) occurred when calling the HeadObject operation: Forbidden

error can also be caused if your system clock is too far off. I was 12 hours in the past and got this error. Set the clock to within a minute of the true time, and error went away.

Answer 3

According to Granting Access to a Single S3 Bucket Using Amazon IAM, the IAM policy may need to be applied to two resources:

1. The bucket proper (e.g. "arn:aws:s3:::4ormat-knowledge-base")
2. All the objects inside the bucket (e.g. "arn:aws:s3:::4ormat-knowledge-base/*")

Yet another tripwire. Damn!

Answer 4

I just got this error because I had an old version of awscli:

Broken:

$ aws --version aws-cli/1.2.9 Python/3.4.0 Linux/3.13.0-36-generic

Works:

$ aws --version aws-cli/1.5.4 Python/3.4.0 Linux/3.13.0-36-generic

Answer 5

You also get this error if the key doesn't exist in the bucket.

Double-check the key -- I had a script that was adding an extra slash at the beginning of the key when it POSTed items into the bucket. So this:

aws s3 cp --region us-east-1 s3://bucketname/path/to/file /tmp/filename

failed with "A client error (Forbidden) occurred when calling the HeadObject operation: Forbidden."

But this:

aws s3 cp --region us-east-1 s3://bucketname//path/to/file /tmp/filename

worked just fine. Not a permissions issue at all, just boneheaded key creation.

Answer 6

I had this error because I didn't attach a policy to my IAM user.

Answer 7

tl;dr: wild card file globbing worked better in s3cmd for me.

As cool as aws-cli is --for my one-time S3 file manipulation issue that didn't immediately work as I would hope and thought it might-- I ended up installing and using s3cmd.

Whatever syntax and behind the scenes work I conceptually imagined, s3cmd was more intuitive and accomodating to my baked in preconceptions.

Maybe it isn't the answer you came here for, but it worked for me.