Force CloudFront distribution/file update

StackOverflow https://stackoverflow.com/questions/1268158

Question

I'm using Amazon's CloudFront to serve static files of my web apps.

Is there no way to tell a cloudfront distribution that it needs to refresh it's file or point out a single file that should be refreshed?

Amazon recommend that you version your files like logo_1.gif, logo_2.gif and so on as a workaround for this problem but that seems like a pretty stupid solution. Is there absolutely no other way?

Was it helpful?

Solution

Good news. Amazon finally added an Invalidation Feature. See the API Reference.

This is a sample request from the API Reference:

POST /2010-08-01/distribution/[distribution ID]/invalidation HTTP/1.0
Host: cloudfront.amazonaws.com
Authorization: [AWS authentication string]
Content-Type: text/xml

<InvalidationBatch>
   <Path>/image1.jpg</Path>
   <Path>/image2.jpg</Path>
   <Path>/videos/movie.flv</Path>
   <CallerReference>my-batch</CallerReference>
</InvalidationBatch>

OTHER TIPS

As of March 19, Amazon now allows Cloudfront's cache TTL to be 0 seconds, thus you (theoretically) should never see stale objects. So if you have your assets in S3, you could simply go to AWS Web Panel => S3 => Edit Properties => Metadata, then set your "Cache-Control" value to "max-age=0".

This is straight from the API documentation:

To control whether CloudFront caches an object and for how long, we recommend that you use the Cache-Control header with the max-age= directive. CloudFront caches the object for the specified number of seconds. (The minimum value is 0 seconds.)

With the Invalidation API, it does get updated in a few of minutes.
Check out PHP Invalidator.

Automated update setup in 5 mins

OK, guys. The best possible way for now to perform automatic CloudFront update (invalidation) is to create Lambda function that will be triggered every time when any file is uploaded to S3 bucket (a new one or rewritten).

Even if you never used lambda functions before, it is really easy -- just follow my step-by-step instructions and it will take just 5 mins:

Step 1

Go to https://console.aws.amazon.com/lambda/home and click Create a lambda function

Step 2

Click on Blank Function (custom)

Step 3

Click on empty (stroked) box and select S3 from combo

Step 4

Select your Bucket (same as for CloudFront distribution)

Step 5

Set an Event Type to "Object Created (All)"

Step 6

Set Prefix and Suffix or leave it empty if you don't know what it is.

Step 7

Check Enable trigger checkbox and click Next

Step 8

Name your function (something like: YourBucketNameS3ToCloudFrontOnCreateAll)

Step 9

Select Python 2.7 (or later) as Runtime

Step 10

Paste following code instead of default python code:

from __future__ import print_function

import boto3
import time

def lambda_handler(event, context):
    for items in event["Records"]:
        path = "/" + items["s3"]["object"]["key"]
        print(path)
        client = boto3.client('cloudfront')
        invalidation = client.create_invalidation(DistributionId='_YOUR_DISTRIBUTION_ID_',
            InvalidationBatch={
            'Paths': {
            'Quantity': 1,
            'Items': [path]
            },
            'CallerReference': str(time.time())
            })

Step 11

Open https://console.aws.amazon.com/cloudfront/home in a new browser tab and copy your CloudFront distribution ID for use in next step.

Step 12

Return to lambda tab and paste your distribution id instead of _YOUR_DISTRIBUTION_ID_ in the Python code. Keep surrounding quotes.

Step 13

Set handler: lambda_function.lambda_handler

Step 14

Click on the role combobox and select Create a custom role. New tab in browser will be opened.

Step 15

Click view policy document, click edit, click OK and replace role definition with following (as is):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
          "cloudfront:CreateInvalidation"
      ],
      "Resource": [
          "*"
      ]
    }
  ]
}

Step 16

Click allow. This will return you to a lambda. Double check that role name that you just created is selected in the Existing role combobox.

Step 17

Set Memory (MB) to 128 and Timeout to 5 sec.

Step 18

Click Next, then click Create function

Step 19

You are good to go! Now on, each time you will upload/reupload any file to S3, it will be evaluated in all CloudFront Edge locations.

PS - When you are testing, make sure that your browser is loading images from CloudFront, not from local cache.

PSS - Please note, that only first 1000 files invalidation per month are for free, each invalidation over limit cost $0.005 USD. Also additional charges for Lambda function may apply, but it is extremely cheap.

Bucket Explorer has a UI that makes this pretty easy now. Here's how:

Right click your bucket. Select "Manage Distributions."
Right click your distribution. Select "Get Cloudfront invalidation list" Then select "Create" to create a new invalidation list. Select the files to invalidate, and click "Invalidate." Wait 5-15 minutes.

If you have boto installed (which is not just for python, but also installs a bunch of useful command line utilities), it offers a command line util specifically called cfadmin or 'cloud front admin' which offers the following functionality:

Usage: cfadmin [command]
cmd - Print help message, optionally about a specific function
help - Print help message, optionally about a specific function
invalidate - Create a cloudfront invalidation request
ls - List all distributions and streaming distributions

You invaliate things by running:

$sam# cfadmin invalidate <distribution> <path>

Just posting to inform anyone visiting this page (first result on 'Cloudfront File Refresh') that there is an easy-to-use+access online invalidator available at swook.net

This new invalidator is:

  • Fully online (no installation)
  • Available 24x7 (hosted by Google) and does not require any memberships.
  • There is history support, and path checking to let you invalidate your files with ease. (Often with just a few clicks after invalidating for the first time!)
  • It's also very secure, as you'll find out when reading its release post.

Full disclosure: I made this. Have fun!

one very easy way to do it is FOLDER versioning.

So if your static files are hundreds for example, simply put all of them into a folder called by year+versioning.

for example i use a folder called 2014_v1 where inside i have all my static files...

So inside my HTML i always put the reference to the folder. ( of course i have a PHP include where i have set the name of the folder. ) So by changing in 1 file it actually change in all my PHP files..

If i want a complete refresh, i simply rename the folder to 2014_v2 into my source and change inside the php include to 2014_v2

all HTML automatically change and ask the new path, cloudfront MISS cache and request it to the source.

Example: SOURCE.mydomain.com is my source, cloudfront.mydomain.com is CNAME to cloudfront distribution.

So the PHP called this file cloudfront.mydomain.com/2014_v1/javascript.js and when i want a full refresh, simply i rename folder into the source to "2014_v2" and i change the PHP include by setting the folder to "2014_v2".

Like this there is no delay for invalidation and NO COST !

This is my first post in stackoverflow, hope i did it well !

In ruby, using the fog gem

AWS_ACCESS_KEY = ENV['AWS_ACCESS_KEY_ID']
AWS_SECRET_KEY = ENV['AWS_SECRET_ACCESS_KEY']
AWS_DISTRIBUTION_ID = ENV['AWS_DISTRIBUTION_ID']

conn = Fog::CDN.new(
    :provider => 'AWS',
    :aws_access_key_id => AWS_ACCESS_KEY,
    :aws_secret_access_key => AWS_SECRET_KEY
)

images = ['/path/to/image1.jpg', '/path/to/another/image2.jpg']

conn.post_invalidation AWS_DISTRIBUTION_ID, images

even on invalidation, it still takes 5-10 minutes for the invalidation to process and refresh on all amazon edge servers

current AWS CLI support invalidation in preview mode. Run the following in your console once:

aws configure set preview.cloudfront true

I deploy my web project using npm. I have the following scripts in my package.json:

{
    "build.prod": "ng build --prod --aot",
    "aws.deploy": "aws s3 sync dist/ s3://www.mywebsite.com --delete --region us-east-1",
    "aws.invalidate": "aws cloudfront create-invalidation --distribution-id [MY_DISTRIBUTION_ID] --paths /",
    "deploy": "npm run build.prod && npm run aws.deploy && npm run aws.invalidate"
}

Having the scripts above in place you can deploy your site with:

npm run deploy

If you are using AWS, you probably also use its official CLI tool (sooner or later). AWS CLI version 1.9.12 or above supports invalidating a list of file names.

Full disclosure: I made this. Have fun!

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top