Access Denied s3cmd from an EC2 machine

StackOverflow https://stackoverflow.com/questions/23245898

Question

I'm trying to use a log rotation configuration for my nginx server that I'm using as a reverse proxy machine located on an EC2 Ubuntu instance.

I want to store those logs on a S3 bucket after a rotation but I'm only getting "access denied, are you sure you keys have ListAllMyBuckets permissions errors" when I'm trying to configure s3cmd tools.

I'm pretty sure that my credentials is correctly configured at IAM, tried at least five different credentials (even the root cred) with the same result. It works fine to list all of my buckets from my local computer with aws cli tools with the same credentials so it puzzles me that I don't have any access just on my EC2 instance.

this is what I run:

which s3cmd
/usr/local/bin/s3cmd

s3cmd --configure --debug

Access Key: **************
Secret Key: *******************************
Encryption password: 
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: False
HTTP Proxy server name:
HTTP Proxy server port: 0

and this is the result 

...
DEBUG: ConnMan.put(): connection put back to pool (http://s3.amazonaws.com#1)
DEBUG: S3Error: 403 (Forbidden)
DEBUG: HttpHeader: x-amz-id-2: nMI8DF+............
DEBUG: HttpHeader: server: AmazonS3
DEBUG: HttpHeader: transfer-encoding: chunked
DEBUG: HttpHeader: x-amz-request-id: 5912737605BB776C
DEBUG: HttpHeader: date: Wed, 23 Apr 2014 13:16:53 GMT
DEBUG: HttpHeader: content-type: application/xml
DEBUG: ErrorXML: Code: 'AccessDenied'
DEBUG: ErrorXML: Message: 'Access Denied'
DEBUG: ErrorXML: RequestId: '5912737605BB776C'
DEBUG: ErrorXML: HostId: 'nMI8DF+............
ERROR: Test failed: 403 (AccessDenied): Access Denied
ERROR: Are you sure your keys have ListAllMyBuckets permissions?

The only thing that is in front of my nginx server is a load balancer, but I can't see why it could interfere with my request. Could it be something else that I've missed?

Was it helpful?

Solution 5

I found out a solution for my problems by deleting all installation of s3cmd. Then made sure that apt-get was up to date and installing it from apt-get again. After my configuration (the same as before) it worked out just fine!

OTHER TIPS

Please check That IAM user permission which keys you are using

Steps would be

  • AWS console go to IAM panel
  • IAM user > Select that User > in the bottom menu 2nd tab is permission

  • attach a user policy

    {
"Version": "2012-10-17",
"Statement": [
 {
  "Effect": "Allow",
  "Action": ["s3:ListAllMyBuckets"],
  "Resource": "arn:aws:s3:::*"
},
{
  "Effect": "Allow",
  "Action": [
    "s3:ListBucket",
    "s3:GetBucketLocation"
  ],
  "Resource": "arn:aws:s3:::YOU-Bucket-Name"

},
{
  "Effect": "Allow",
  "Action": [
    "s3:PutObject",
    "s3:GetObject",
    "s3:DeleteObject"
  ],
  "Resource": "arn:aws:s3:::YOU-Bucket-Name/*"


}
]
}

Let me know how it goes

Please dont trust the --configure switch:

i was facing the same problem. it was showing 403 in --configure but at the end i saved the Settings and then tried:

ERROR: Test failed: 403 (AccessDenied): Access Denied
Retry configuration? [Y/n] n
Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'

# s3cmd put MyFile s3://MyBucket/

& it worked..

s3cmd creates a file called .s3cfg in your home directory when you set this up. I would make sure you put this file somewhere where your logrotate script can read this, and use the -c flag.

For example to upload the logfile.txt file to the logbucket bucket:

/usr/local/bin/s3cmd -c /home/ubuntu/.s3cfg put logfile.txt s3://logbucket

What is the version of s3cmd you are using?

I tried it using s3cmd 1.1, it seems s3cmd 1.1 does not work with IAM roles.

But someone says s3cmd 1.5 alpha2 has support for IAM roles.(http://t1983.file-systems-s3-s3tools.file-systemstalk.info/s3cmd-1-5-0-alpha2-iam-roles-supportincluded-t1983.html)

I have tried s3cmd 1.5 beta1(https://github.com/s3tools/s3cmd/archive/v1.5.0-beta1.tar.gz), it works fine with IAM roles.

So there are two ways to access s3 bucket of s3cmd:

  1. Using access key and secret key `

    you need to set a config file in /root/.s3cfg(default path) as bellow

    access_key=xxxxxxxx secret_key=xxxxxxxxxxxxxxxxxxxx

    Note that just set above two key-value in .s3cfg, no need other keys.

    `
  2. Using IAM add s3 policy with s3cmd > 1.5 alph2. `

    you need add a IAM to ec2 instance, this role may has a policy as bellow

    { "Effect": "Allow", "Action": [ "s3:" ], "Resource": "" } `

I also had a similar problem. Even after associating my EC2 instance to an IAM role with s3 full access policy, my s3cmd was failing as there wasn't any .s3cfg file in it. I fixed by updating the version of my s3cmd.

sudo pip install s3cmd==1.6.1

Did the trick!

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top