Take a look at RFC 2616 - Caching in HTTP.
You even don't need to download file and check the Modified Time.
You can easily send a request with HEAD
method. (not POST
or GET
) then check some HTTP Response headers like:
Etag
You can save ETag for each entity and save it somewhere in your server. Then check if ETag has changed, send a
GET
request to retrieve new file.Last-Modified
Using This field is much easier. Compare your last update time with this field's value
Expires
You can plan to send request in future by this time.
Existence of each field depends on the webserver's configuration and the way they made RSS Generator. But usually one of these fields exists.
Also there is alternative way to check content's length by checking the Content-Length
field with your file's length which exists on your server.