HTTP streaming connection (SSE) client disconnect not detected with Sinatra/Thin on Heroku

Question

I am attempting to deploy a Sinatra streaming SSE response application on the Cedar stack. Unfortunately while it works perfectly in development, once deployed to Heroku the callback or errback never get called when a connection is called, leading to the connection pool getting filled up with stale connections (that never time out because data is still being sent to them on the server side.)

Relvant info from the Heroku documentation:

Long-polling and streaming responses

Cedar supports HTTP 1.1 features such as long-polling and streaming responses. An application has an initial 30 second window to respond with a single byte back to the client. However, each byte transmitted thereafter (either received from the client or sent by your application) resets a rolling 55 second window. If no data is sent during the 55 second window, the connection will be terminated.

If you’re sending a streaming response, such as with server-sent events, you’ll need to detect when the client has hung up, and make sure your app server closes the connection promptly. If the server keeps the connection open for 55 seconds without sending any data, you’ll see a request timeout.

This is exactly what I would like to do -- detect when the client has hung up, and close the connection promptly. However, something about the Heroku routing layer seems to prevent Sinatra from detecting the stream close event as it would normally.

Some sample code that can be used to replicate this:

require 'sinatra/base'

class MyApp < Sinatra::Base

  set :path, '/tmp'
  set :environment, 'production'

  def initialize
    @connections = []

    EM::next_tick do
      EM::add_periodic_timer(1) do
        @connections.each do |out|
          out << "connections: " << @connections.count << "\n"
        end
        puts "*** connections: #{@connections.count}"
      end
    end

  end

  get '/' do
    stream(:keep_open) do |out|
      @connections << out
      puts "Stream opened from #{request.ip} (now #{@connections.size} open)"

      out.callback do
        @connections.delete(out)
        puts "Stream closed from #{request.ip} (now #{@connections.size} open)"
      end
    end
  end

end

I've put a sample app up at http://obscure-depths-3413.herokuapp.com/ using this code that illustrates the problem. When you connect, the amount of connections will increment, but when you disconnect they never go down. (Full source of demo with Gemfile etc is at https://gist.github.com/mroth/5853993)

I'm at wits end trying to debug this one. Anyone know how to fix it?

P.S. There appears to have been a similar bug in Sinatra but it was fixed a year ago. Also this issue only occurs on production in Heroku, but works fine when run locally.

P.S.2. This occurs when iterating over the connections objects as well, for example adding the following code:

EM::add_periodic_timer(10) do
  num_conns = @connections.count
  @connections.reject!(&:closed?)
  new_conns = @connections.count
  diff = num_conns - new_conns
  puts "Purged #{diff} connections!" if diff > 0
end

Works great locally, but the connections never appear as closed on Heroku.

Answer 1

An update: after working directly with the Heroku routing team (who are great guys!), this is now fixed in their new routing layer, and should work properly in any platform.

Answer 2

I would do this check by hand sending, in a periodic time, alive signal where the client should respond if the message was received.

Please, look at this simple chat implementation https://gist.github.com/tlewin/5708745 that illustrate this concept.

The application communicates with the client using a simple JSON protocol. When the client receive the alive: true message, the application post back a response and the server store the last communication time.