How do I configure Tomcat and HAProxy to work with the WebSocket Framework Atmosphere?

Question

I have a Java application running on Tomcat7, which uses the Atmosphere Framework. Atmosphere is a Websocket Framework. I used a sample application from the Atmosphere Sample Chat. My problem is that I did not get the connectors as well as the proper configuration for my Websocket Atmosphere application running.

The questions I have are:

1. How do I have to configure my HAProxy?
2. How do I have to configure my Tomcat server?
3. How do I have to configure Tomcat for APR and for the NIO connector?
4. Do I have to change something in my app (i.e., my web.xml)?
5. How do I have to fix the errors below?

This is my HAProxy config:

defaults
mode http
log global
option httplog
option  http-server-close
option  dontlognull
option  redispatch
option  contstats
retries 3
backlog 10000
timeout client      25s
timeout connect     5s
timeout server      25s
# timeout tunnel available in ALOHA 5.5 or HAProxy 1.5-dev10 and higher
#   timeout tunnel  3600s
timeout http-keep-alive 1s
timeout http-request        15s
timeout queue        30s
timeout tarpit      60s
default-server inter 3s rise 2 fall 3
option forwardfor


frontend ft_web
  bind *:80 name http
  maxconn 60000

  acl host_testhostdev  hdr(host) -i www.testhost.com

  use_backend apache-8080 if host_testhostdev

  default_backend apache-8080


backend apache-8080
  balance roundrobin

  server websrv1 localhost:8080 maxconn 8000 weight 10 cookie websrv1 check

The web.xml part for my Atmosphere Servlet:

<async-supported>true</async-supported>
<init-param>
   <param-name>org.atmosphere.useNative</param-name>
   <param-value>true</param-value>
</init-param>   

<init-param>
<param-name>org.atmosphere.cpr.AtmosphereInterceptor</param-name>
    <param-value>org.atmosphere.interceptor.HeartbeatInterceptor,org.atmosphere.interceptor.AtmosphereResourceLifecycleInterceptor,org.atmosphere.client.TrackMessageSizeInterceptor</param-value>
</init-param>

<init-param>
   <param-name>org.atmosphere.cpr.broadcaster.shareableThreadPool</param-name>
   <param-value>true</param-value>
</init-param>

<init-param>
   <param-name>org.atmosphere.cpr.sessionSupport</param-name>
   <param-value>true</param-value>
</init-param>
<init-param>
   <param-name>org.atmosphere.cpr.broadcasterCacheClass</param-name>
   <param-value>org.atmosphere.cache.UUIDBroadcasterCache</param-value>
</init-param>

What I changed in Tomcat server.xml

!--
<Connector port="8080" protocol="HTTP/1.1"
           connectionTimeout="20000"
           redirectPort="8443" />
 -->

<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" redirectPort="8443"/>  

These are the errors I get:

 ERROR cpr.AtmosphereFramework  - If you have more than one Connector enabled, make sure they both use the same protocol, e.g NIO/APR or HTTP for all. If not, org.atmosphere.container.BlockingIOCometSupport will be used and cannot be changed.

and

 2013-08-02 11:03:11,384 [http-nio-8080-exec-26] ERROR test.ReflectionService  - 
 Service method exception in publishDataCreatedCount in service  ClientNotificationUpdateService
 java.lang.NullPointerException
at org.atmosphere.cpr.AtmosphereRequest.setAttribute(AtmosphereRequest.java:569)
at org.atmosphere.cpr.DefaultBroadcaster.executeAsyncWrite(DefaultBroadcaster.java:922)
at org.atmosphere.util.SimpleBroadcaster.queueWriteIO(SimpleBroadcaster.java:192)
at org.atmosphere.cpr.DefaultBroadcaster.deliverPush(DefaultBroadcaster.java:768)
at org.atmosphere.cpr.DefaultBroadcaster.push(DefaultBroadcaster.java:666)
at org.atmosphere.util.SimpleBroadcaster.broadcast(SimpleBroadcaster.java:133)
at test.ClientUpdateService.sendToBoard(ClientUpdateService.groovy:46)
at test.ClientNotificationUpdateService.publishDataCreatedCount(ClientNotificationUpdateService.groovy:179)
at test.ReflectionService.callByConvention(ReflectionService.groovy:59)
at test.ClientUpdateService.methodMissing(ClientUpdateService.groovy:65)
at test.NotificationService.create(NotificationService.groovy:206)
at test.HomeController.addAction(HomeController.groovy:402)
at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:195)
at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)

and this one if I access my app with IE8 and IE9:

2013-08-08 00:56:19,631 [http-nio-8080-exec-9] ERROR cpr.AtmosphereFramework  - AtmosphereFramework exception
java.lang.IllegalStateException: Not supported.
    at org.atmosphere.cpr.AtmosphereRequest.startAsync(AtmosphereRequest.java:594)
    at org.atmosphere.container.Servlet30CometSupport.suspend(Servlet30CometSupport.java:130)
    at org.atmosphere.container.Servlet30CometSupport.service(Servlet30CometSupport.java:105)
    at org.atmosphere.container.Tomcat7Servlet30SupportWithWebSocket.doService(Tomcat7Servlet30SupportWithWebSocket.java:65)
    at org.atmosphere.container.TomcatWebSocketUtil.doService(TomcatWebSocketUtil.java:87)
    at org.atmosphere.container.Tomcat7Servlet30SupportWithWebSocket.service(Tomcat7Servlet30SupportWithWebSocket.java:61)
    at org.atmosphere.cpr.AtmosphereFramework.doCometSupport(AtmosphereFramework.java:1594)
    at org.atmosphere.cpr.AtmosphereServlet.doPost(AtmosphereServlet.java:176)
    at org.atmosphere.cpr.AtmosphereServlet.doGet(AtmosphereServlet.java:162)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)
2013-08-08 00:56:19,670 [http-nio-8080-exec-2] ERROR cpr.AtmosphereFramework  - AtmosphereFramework exception
java.lang.IllegalStateException: Not supported.
    at org.atmosphere.cpr.AtmosphereRequest.startAsync(AtmosphereRequest.java:594)
    at org.atmosphere.container.Servlet30CometSupport.suspend(Servlet30CometSupport.java:130)
    at org.atmosphere.container.Servlet30CometSupport.service(Servlet30CometSupport.java:105)
    at org.atmosphere.container.Tomcat7Servlet30SupportWithWebSocket.doService(Tomcat7Servlet30SupportWithWebSocket.java:65)
    at org.atmosphere.container.TomcatWebSocketUtil.doService(TomcatWebSocketUtil.java:87)
    at org.atmosphere.container.Tomcat7Servlet30SupportWithWebSocket.service(Tomcat7Servlet30SupportWithWebSocket.java:61)
    at org.atmosphere.cpr.AtmosphereFramework.doCometSupport(AtmosphereFramework.java:1594)
    at org.atmosphere.cpr.AtmosphereServlet.doPost(AtmosphereServlet.java:176)
    at org.atmosphere.cpr.AtmosphereServlet.doGet(AtmosphereServlet.java:162)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

Answer 1

Okay here is what I would do. Install the APR Connector on your Tomcat server. This will speed up the production environment. Download APR and follow the install instructions to compile it on your os. You do not need the NIO Connector anymore.

Edit your server.xml:

<Connector port="8080" 
            protocol="org.apache.coyote.http11.Http11AprProtocol" 
            URIEncoding="UTF-8" 
            maxThreads="250"
            redirectPort="8443" />  

Do not use the org.atmosphere.interceptor.AtmosphereResourceLifecycleInterceptor this causes errors for me.

Your HAProxy config is fine. You should also use the atmosphere-runtime-native.jar instead of the atmosphere-runtime.jar.

Your errors might occur because you connect with a browser that does not support websockets to your application which uses websockets.