Necessary s3cmd S3 permissions for PUT/Sync

Question

In moving to AWS EC2, I want to restrict my instances' user permissions for good reason. One thing the instances need to do is access files on S3 and write files there. However, I cannot find any way to achieve this without giving all permissions to that user.

s3cmd allows me to call "ls" and "du" on the s3 buckets I gave the policy permission to, but always fails with a 403 error when trying to PUT/sync with one of these folders. If I use my root credentials, the transfer goes right through.

So, I don't get why if I give all permissions to the user for said buckets, it cannot PUT, but if I give it arn:aws:s3:::* (all buckets) then it can. Makes no sense to me.

Anyone else ever dealt with this before?

Answer 1

Try something like this. I think the problem is that you need s3:ListAllMyBuckets and s3:ListBuckets for the s3cmd to work. Not sure why but it wont work unless it can get a list of the buckets. I had the same problem the first time i tried to use permissions with s3cmd and this was the solution.

{
  "Statement": [
    {
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::*"
    },
    {
      "Action": [ 
          "s3:ListBucket", 
          "s3:PutObject",
          "s3:PutObjectAcl"
      ],
      "Effect": "Allow",
      "Resource": [
          "arn:aws:s3:::bucket/path", 
          "arn:aws:s3:::bucket/path/*"
      ]
    }
  ]
}

Edit I've added the s3:PutObjectAcl action which is required for newer versions of s3cmd as stated by Will Jessop below.

Answer 2

bwight's answer is almost right (it probably used to be for older versions of s3cmd), but I need to add a s3:PutObjectAcl to get it to work:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt123456",
      "Effect": "Allow",
      "Action": [
        "s3:ListAllMyBuckets"
      ],
      "Resource": [
        "arn:aws:s3:::*"
      ]
    },
    {
      "Sid": "Stmt123457",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": [
        "arn:aws:s3:::bucketname",
        "arn:aws:s3:::bucketname/*"
      ]
    }
  ]
}

Answer 3

I was trying to do big file uploads and the policy wasn't working well for me, I ended adding the next policy to the user:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1397834652000",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "Stmt1397834745000",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:GetObject",
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:GetObjectVersionAcl"
            ],
            "Resource": [
                "arn:aws:s3:::my_bucket",
                "arn:aws:s3:::my_bucket/*"
            ]
        }
    ]
}

where my_bucket is the bucket where I need to manage files though s3cmd

Answer 4

In case you are giving access to a subfolder (as in the original answer of /bucket-name/path/) and not the entire bucket, the ListBucket action requires a bit more specificity:

{
    "Sid": "AllowListingOfFilesInFolder",
    "Effect": "Allow",
    "Action": [
        "s3:ListBucket"
    ],
    "Resource": [
        "arn:aws:s3:::bucket-name"
    ],
    "Condition": {
        "StringLike": {
            "s3:prefix": [
                "path/*"
            ]
        }
    }
}

I believe it works also with the original answer in case you provide access to the entire bucket.