Content CDNs and dynamic content

Question

I am curious as to how dynamic web applications play with content delivery networks such as cloudflare, amazon cloudfront, etc. I understand the concept of a cdn obtaining a cache of your static pages and then serving those pages from their various servers relative to where the request is coming from. However when it comes to applications that are database heavy that require user sessions, allow for searches, and other user interaction is a cdn still useful?

If the answer is yes, does the cdn crawl your server and create caches of all of the pages that could be created dynamically?

If this is true then how is sensitive data handled? (For example if your application is pulling profile information specific to a user that only that user should have access to.)

Answer 1

CDNs are used mostly for really static content: content which is the same for every or most users, and which will probably stay unchanged for days, weeks or months. CDNs are ideal for CSS files, JavaScript or images: they don't change much, they are usually cached for a year on client side anyway, and when they change, propagating the change to CDNs and clients is painless (using a revision number in the URI, such as global.css?r=514 is one of the ways to do it).

CDNs can also be used for content which is dynamic, but:

• Either is identical for every user,
• Or stays unchanged for days, weeks or months.

The more the content is dynamic, the less beneficial is the CDN. For example, you can use a CDN for a blog where content changes rarely (say a new blog post per week). While being dynamic, the content is the same for every user and doesn't need to be refreshed too often (for example, caching all blog posts for a hour is OK, because not being able to see the most recent post for an hour is acceptable in this case).

If the content is more dynamic, it is processed directly by the application servers, not the CDNs. By the way, crawling the website would be practically impossible, because the content will change during the crawling process or vary from user to user. For example, there is no way for a CDN to cache my Amazon purchasing page: not only do I need the up-to-date version of the webpage, containing my last purchase I've done five seconds ago, but there is no benefit of having a CDN at this level.

Note that since the primary goal of a CDN is to be close to the users (i.e. a user from New Delhi shouldn't be waiting for a server in Nevada to process his request), the same technique is used for large sites as well: dynamic requests are often processed by servers which are close enough to the users, when the company can afford building several data centers around the world.