Azure web role runnig Java returns 500

Question

I am new to Java/Apache tomcat

I needed to build a web service in azure cloud services, using java. The web service is running on apache tomcat 7, jdk 1.7 and communicating a mongo-db that is running on an azure vm.

The service is used as a "proxy" to a bit complex key-value data that is stored in the mongodb, and it should be massively requested in the near future.

It worked fine on development, buy when starting to use it on production (~100K requests a day) it started returning 500 (502 and 503) after a while. Restarting the cloud service server is fixing this issue but after a day or two it happens again.

It does not seems to be something that is related to a big load because the CPU stats are good (no more than 70%, mostly 30%).

The problem is not related to the mongodb, since its CPU is less than 2% and when restarting the java role it all works good.

Am I missing out something in the server's setting ?

Answer 1

Sounds suspiciously like a memory leak. Apply standard troubleshooting techniques - Azure is no different to running Java than any other environment. You need to do some load testing in your development environment that simulates your ~100k / 24 hour requests (that's 1.15 requests a second which doesn't sound all that high).