Logging Total Data Downloads In Wget

Question

I'm looking to create a running log of all data transfers within wget on my Linux system (CentOS 6.5).

I'd like to know how much data I've downloaded with wget.

Answer 1

I guess you could create an alias for wget to always log to a global file:

alias wget='wget -a ~/tmp/wget.log '

Then for every downloaded file there should be a log like this:

Length: 82651 (81K) [text/html]
Saving to: ‘index.html’

     0K .......... .......... .......... .......... .......... 61% 62.5K 0s
    50K .......... .......... ..........                      100%  151K=1.0s

Last-modified header missing -- time-stamps turned off.
2013-12-27 10:30:58 (80.4 KB/s) - ‘index.html’ saved [82651/82651]

So you could use your favorite text processing tools to extract the downloaded bytes, for example:

sed -ne 's?.* saved \[\([0-9][0-9]*\)/.*?\1?p' ~/tmp/wget.log

or

awk '/^Length:/ {print $2}' ~/tmp/wget.log