We did some further debugging on slow apache file upload operations and got the root cause of slowness as described below:
We were using mod_jk connector for making apache talk to tomcat servers and it was having two problems:
- Mod_jk connectors log level was set to debug by default , which was making all uploaded data bytes to be written to jk.log file too, and hence causing a slowness, after correcting it time came down to 1 minute 30 seconds as compared to 4 minutes 30 seconds initially observed by us. Better but not enough!
- After further debugging we found that connector protocol behind mod_jk, i.e. AJP has an inherent limitation of max packet size of 64 KB which is lower than what can consume on our server side 16 MB I guess after seeing the configuration on test setup. So pipe was narrow at mod_jk side and hence was affecting end to end upload speed.
After this we did some tests using http connector (via mod_proxy) and load balancer (via mod_proxy_balancer) and have achieved excellent results