Twitter4j streaming API fails with "connect timed out" error

Question

We are using twitter4j for streaming twitter data, but getting "connect timed out" error. But if we use curl with the authorization information and execute from the same system, it works fine and we get the tweets.

Below is the java code snippet:

ArrayList<String> track = new ArrayList<String>();
track.add("#usa");
String[] trackArray = track.toArray(new String[track.size()]);
twitterStream.filter(new FilterQuery(0, null, trackArray));

Below is the stack trace, the google links provided in the error message do not give much information.

Relevant discussions can be found on the Internet at:
    http://www.google.co.jp/search?q=944a924a or
    http://www.google.co.jp/search?q=24fd66dc
TwitterException{exceptionCode=[944a924a-24fd66dc 944a924a-24fd66b2], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=3.0.5}
    at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:177)
    at twitter4j.internal.http.HttpClientWrapper.request(HttpClientWrapper.java:61)
    at twitter4j.internal.http.HttpClientWrapper.post(HttpClientWrapper.java:98)
    at twitter4j.TwitterStreamImpl.getFilterStream(TwitterStreamImpl.java:304)
    at twitter4j.TwitterStreamImpl$7.getStream(TwitterStreamImpl.java:292)
    at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:462)
Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:310)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:176)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:163)
    at java.net.Socket.connect(Socket.java:546)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:169)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:409)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:530)
    at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:289)
    at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:346)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:755)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:858)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:250)
    at twitter4j.internal.http.HttpClientImpl.request(HttpClientImpl.java:135)
    ... 5 more
20254 [Twitter Stream consumer-2[Establishing connection]] INFO  twitter4j.TwitterStreamImpl  - Waiting for 250 milliseconds

Below is the curl command

curl --get 'https://stream.twitter.com/1.1/statuses/filter.json' --data 'track=usa' --header 'Authorization: OAuth oauth_consumer_key="hidden_value", oauth_nonce="hidden_value", oauth_signature="hidden_value", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1388654077", oauth_token="hidden_value", oauth_version="1.0"'

We use a proxy server to connect to internet and are using cntlm with the user credentials along with proxy server details specified in cntlm.

Answer 1

The issue is resolved.

Earlier I was setting the proxy host and port using below code which didn't work:

System.getProperties().put("http.proxyHost", "proxy");
System.getProperties().put("http.proxyPort", "8080");

But it works by setting the proxy using ConfigurationBuilder

cb.setHttpProxyHost("proxy");
cb.setHttpProxyPort(8080);

Answer 2

You can also set proxy host and port by using twitter4j properties in flume-ng command as below

$ flume-ng agent -n TwitAgent -c conf -f /usr/hadoop/FLUME/apache-flume-1.3.1-bin/conf/flume.conf -Dtwitter4j.http.proxyHost=www-proxy.example.com -Dtwitter4j.http.proxyPort=80