Why is my Rack app hanging?

Question

I've written an open source Rack app and core lirary to stream stock quotes from Tradeking.com. Here's the rack app: https://github.com/chaddjohnson/trading_websocket_service. And here's the core library: https://github.com/chaddjohnson/trading_core.

I modified trading_core/lib/trading_core/quote_streamer/tradeking.rb to output streamed data to the console, as so:

@http.stream do |data|
  puts data

After a few hours of it streaming and outputting to the console, it simply freezes. I have absolutely no idea why. I've done my best to ensure that any loops are not infinite. I've been debugging this for over a week now.

Any ideas why this is happening?

Here are the main files where I think this could be happening:

• https://github.com/chaddjohnson/trading_core/blob/master/lib/trading_core/client_data_broker.rb
• https://github.com/chaddjohnson/trading_core/blob/master/lib/trading_core/quote_streamer/tradeking.rb
• https://github.com/chaddjohnson/trading_websocket_service/blob/master/lib/web_socket_service.rb

Answer 1

Glad I could push you in the right direction!

My first hunches were about open connections:

You might need to make sure you close the connection: @http.close in an ensure block at the end of the method you referenced

Then on to some forensics:

Sounds like you'll have to dig into the ruby process you're running to figure out what's choking it up. Depending on what OS you're on of course, on mac and linux you can use tools like gdb, strace, dtrace, etc. For example: ruby.dzone.com/articles/debugging-stuck-ruby-processes

We found that select was being called a lot which led us to keep our suspicion on the connections. Turns out the problem was in reusing a dead connection via @api ||= ...init code...

Glad you were able to find what it was! And thanks for posting your solution!

Answer 2

On the first look at your code I could imagine two reasons for your issues:

1. You are hitting rate limits with Tradeking (maybe using more than 256 symbols or multiple instances of your script (they don't like that) or updating your symbols too fast (changing them)
2. As you are parsing every incoming message you might "choke" your stream. I see that a lot with people using Twitter streams, parsing the results and writing them to a DB. It's just too slow and then starts blocking.