It was internal network proxy which was for some reason dropping the request after 81.4 MB (85,363,075 bytes) were transfered. But the weird thing is webclient was not throwing any exception and was pretending that download was successful.
Fix is to null the proxy so request is processed without going through proxy.
$client.proxy=$null # get rid of proxy
$client.DownloadFile( $Url, $destination )