Tweetstream stops working

Question

I'm connecting to the twitter API using the tweetstream gem as shown here. It works fine, but when I stop and restart the stream as I'm changing the way I handle the stream, after about 5 restarts over 2 hours or so the stream stops giving me updates, and the on_reconnect event keeps happening every 30 seconds or so.

I'm thinking this is because this triggers rate limiting, or because the previous connections are not killed before starting the new one. Is there a proper way to do this?

Answer 1

I'm not exactly sure as to the cause of my problem, but this is what I've done:

• Update the libraries (bundle update). The discussion on the github issues seemed to imply that the underlying EventMachine gem may have had a bug causing connection issues, which was fixed in the latest version.

• Make sure your system time is realistic. At least once my system time, which wasn't syncing itself with an internet clock, crept more than 30 minutes away from the real time, and my request tokens were being rejected as being invalid (expired, or signed in the future).

• Wrap the stream reader in a Daemon. Give the daemon the {:monitor => true} option, so that it will restart the job if it ever dies. This happens several times a week on my server with a timeout error (I'm guessing API-side service disruptions, or my own internet connection bugging out). This is more of a band-aid because it shouldn't really die in the first place, but it pushes this issue far enough down on my priority list that I'm not currently worrying about it.