curl: how to not overwrite existing file? [closed]

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.

---

This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.

Closed 9 years ago.

Improve this question

Given a +350MB file online ETOPO1_Ice_g_geotiff.zip

Within a script, the following curl command is currently used for downloading :

curl  -o ../data/ETOPO1/ETOPO1.zip \
  'http://www.ngdc.noaa.gov/mgg/global/relief/ETOPO1/data/ice_surface/grid_registered/georeferenced_tiff/ETOPO1_Ice_g_geotiff.zip'

Given the target size and downloading time (~20min), I want to download it the first time only. Then, when I restart the whole script, the command should notice the file as already there, and NOT redownloading it.

Also, when the output -o already exist, how to NOT restart the download/overwrite the file ?

Note: preference for curl, but other utilities welcome (wget, ...)

Answer 1

Use the command-line option -C <offset>:

Continue/Resume a previous file transfer at the given offset. (...) Use "-C -" to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

(Documentation)

In your exact case:

curl \
  -o ../data/ETOPO1/ETOPO1.zip \
  -C - \
  'http://www.ngdc.noaa.gov/mgg/global/relief/ETOPO1/data/ice_surface/grid_registered/georeferenced_tiff/ETOPO1_Ice_g_geotiff.zip'