Boto 403 AccessDenied Exception with IAM user credentials, Works in Cyberduck and AWS web console

Question

I have found lot of questions regarding this on stackoverflow but none solved my problem. After lot of googling still i am facing AccessDenied Exception:

<Error>
<Code>AccessDenied</Code>
</Message><RequestId>ADF9C0DE6C86DF4F</RequestId>
<HostId>JwQLkNB0LuJvh0jwrsJe9wazxLsd+hrZ2qwvjCvmXYd2A/ckCrsotRMHm</HostId>
</Error>

Here are my policy docs for user and group:

User Policy:

{
"Statement":[
  {
     "Sid":"AllowListBucketIfSpecificPrefixIsIncludedInRequest",
     "Action":"s3:*",
     "Effect":"Allow",
     "Resource":["arn:aws:s3::: mybucket", "arn:aws:s3:::mybucket/*"],
     "Condition":{
        "StringLike":{"s3:prefix":["Development/*"]
        }
     }
  },
  {
    "Sid":"AllowUserToReadWriteObjectDataInDevelopmentFolder", 
    "Action":"s3:*",
    "Effect":"Allow",
    "Resource":["arn:aws:s3::: mybucket/Development/*"]
  },
  {
     "Sid": "ExplicitlyDenyAnyRequestsForAllOtherFoldersExceptDevelopment",
     "Action": ["s3:ListBucket"],
     "Effect": "Deny",
     "Resource": ["arn:aws:s3::: mybucket", "arn:aws:s3::: mybucket/*"],
     "Condition":{  "StringNotLike": {"s3:prefix":["Development/*"] },
                    "Null"         : {"s3:prefix":false }
      }
  }
]
}

Group Policy:

{
"Statement": [
{
  "Sid": "AllowGroupToSeeBucketListAndAlsoAllowGetBucketLocationRequiredForListBucket",
  "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
  "Effect": "Allow",
  "Resource": ["arn:aws:s3:::*"]
},
{
  "Sid": "AllowRootLevelListingOfCompanyBucket",
  "Action": ["s3:ListBucket"],
  "Effect": "Allow",
  "Resource": ["arn:aws:s3::: mybucket", "arn:aws:s3::: mybucket/*"],
  "Condition":{
      "StringEquals":{"s3:prefix":[""]}
   }
},
{
  "Sid": "RequireFolderStyleList",
  "Action": ["s3:ListBucket"],
  "Effect": "Deny",
  "Resource": ["arn:aws:s3:::*"],
  "Condition":{
      "StringNotEquals":{"s3:delimiter":"/"}
   }
 },
{
  "Sid": "ExplictDenyAccessToPrivateFolderToEveryoneInTheGroup",
  "Action": ["s3:*"],
  "Effect": "Deny",
  "Resource":["arn:aws:s3:::mybucket/Private/*"]
},
{
  "Sid": "DenyListBucketOnPrivateFolder",
  "Action": ["s3:ListBucket"],
  "Effect": "Deny",
  "Resource": ["arn:aws:s3:::*"],
  "Condition":{
      "StringLike":{"s3:prefix":["Private/"]}
   }
}
]
}

Created a user with username - testuser then got access_key and secret_access_key for this IAM user. Now i am able to access mybucket and its subfolder using aws web console and cyberduck.

But whenever i am trying to access using boto , getting AccessDenied Exception (Error 403).

Boto Code:

<!-- language: python -->
from boto.s3.connection import S3Connection
connect = S3Connection('_______________________','_____________________')
# Without Validate
bucket = conn.get_bucket('mybucket', validate=False) #here got bucket object
bucket.get_key('one/two/three.png') # AccessDenied


#With Validate
bucket = conn.get_bucket('mybucket') #AccessDenied

Even i faced same problem when i was trying to use boto-rsync.

Any Suggestions ??

Answer 1

Error 403 means Access Denied so there is a authentication problem. To analyze the API call and the response one can use the following line:

boto.set_stream_logger('boto')

some points that I have noticed:

• the Group and User Rules are okay, with removed leading space in front of "mybucket"
• the first directory name is "Development" instead of "one"
• "Without Validate" means access file directly

The following code works fine:

import boto
conn = boto.connect_s3("id","secret")
bucket = conn.get_bucket('mybucket', validate=False)
bucket.get_key('Development/two/three.png')
# <Key: mybucket,Development/two/three.png>

But i am new to IAM, and it seems "With Validate" first tries to read "/mybucket/" but it is denied via User Policy ExplicitlyDenyAnyRequestsForAllOtherFoldersExceptDevelopment.

edited to comment "to access all keys inside Development" try this::

list = bucket.list("Development/",delimiter="/")
for key in list:
    print key.name