Not sure why but this seemed to be a network issue. My server was really bottlenecking. I restarted the services and increased Tomcat's memory. Once I did that then the issue went away. Additionally the download manager was only calling a couple files at a given time.
Appears to be a network issue and has been solved.