Sync local file with HTTP server location (in Python)

Question

I have an HTTP server which host some large file and have python clients (GUI apps) which download it.
I want the clients to download the file only when needed, but have an up-to-date file on each run.

I thought each client will download the file on each run using the If-Modified-Since HTTP header with the file time of the existing file, if any. Can someone suggest how to do it in python?

Can someone suggest an alternative, easy, way to achieve my goal?

Answer 1

It seems to me the easiest solution is hosting the file in mercurial and using mercurial api to find the file's hash, downloading the file if the hash has changed. Calculating the hash can be done as the answer to this question; for downloading the file urllib will be enough.

Answer 2

You can add a header called ETag, (hash of your file, md5sum or sha256 etc ), to compare if two files are different instead of last-modified date

Answer 3

I'm assuming some things right now, BUT.. One solution would be to have a separate HTTP file on the server (check.php) which creates a hash/checksum of each files you're hosting. If the files differ from the local files, then the client will download the file. This means that if the content of the file on the server changes, the client will notice the change since the checksum will differ.

do a MD5 hash of the file contents, put it in a database or something and check against it before downloading anything.

Your solution would work to, but it requires the server to actually include the "modified" date in the Header for the GET request (some server softwares does not do this).

I'd say putting up a database that looks something like:

[ID] [File_name] [File_hash]

0001 moo.txt asd124kJKJhj124kjh12j