I found the solution ! The application was working fine with Firefox browser and not working with IE and Chrome. ARR uses a cookie to enable Client Affinity. The Cookie name will be used to set the cookie on the client. That said, the client must accept cookies for client affinity to work properly. Default cookie name is ARRAffinity.
To browse the application I was using the url servername/appname. ARRAffinity Cookie was not getting created when I am browsing the application from IE and Chrome. The cookie got created and the application was working fine when browsed the site using servername.domainname/appname