Using If-Modified-Since header for dynamically generated remote files

Question

Our web server regularly downloads images from other web servers. To prevent our server having to download the same image every day even if it has not changed, I plan to store the Last-Modified header when the image downloads and then put that date in the If-Modified-Since header of subsequent requests for the same file.

I have this working fine except when the remote file is generated on-the-fly when requested (e.g. if it generates a certain sized version for the web when requested from separate original file). In this case, the Last-Modified header is the date that the remote server responds to the request so the stored Last-Modified header from the previous download will always be earlier that ones for subsequent requests so the image will always get downloaded and I'll never get the 304 Not Modified status code.

So, is there a way to reduce the download frequency when the remote server is serving up images that are generated on the fly?

It sounds to me like this is not possible, but I thought I'd ask anyway.

Answer 1

If you can create some form of hash for the the images use ETags. Your server will have to check the If-None-Match request header against the hash and if they match you can return a 304 response.

Clients will still send Last-Modified but if your hashing method does not generate many collision you should be able to ignore it and just match the ETags.