Resume a webpage .read() with urlib

Question

Here is my code :

headers={'User-Agent': 'Mozilla/5.0'}
req=Request(url,headers=headers)
file=open(name,'wb')
file.write(urlopen(req).read())
file.close()

but when i get an exception and want to redownload the file i have to download from the begining ; and HTTPResponse don't have seek method . How can i resume my download ? Thank you !

Answer 1

You can use the byte serving feature of HTTP 1.0. Here you add the range header to your request. For example adding

Range: bytes=9500-

will download the file from the 9500th Byte. So you would do something like this in the end:

1. See how much bytes you have already downloaded.
2. Start the download from the first missing byte and append the output to the file.

Notes:

• The server need to support this technique (is not always the case).
• You can use wget which has already a support for continuing the download (see parameter --continue)