How to handle LinqtoTwitter Rate(v2.1) limit exceeded Error

Question

1 .Hi SO, I have a created a class for fetching user's tweets from twitter with the help of screen name. My problem is I'm getting rate limit exceeded very frequently.

2 .I had created table for screen name in which I'm saving all screen names and

3 .I had created another table to store user's tweets.

Below is my Code:

 public List<TwitterProfileDetails> GetAllTweets(Func<SingleUserAuthorizer> AuthenticateCredentials,string screenname)
    {
        List<TwitterProfileDetails> lstofTweets = new List<TwitterProfileDetails>();
        TwitterProfileDetails details = new TwitterProfileDetails();
        var twitterCtx = new LinqToTwitter.TwitterContext(AuthenticateCredentials());
        var helpResult =
            (from help in twitterCtx.Help
             where help.Type == HelpType.RateLimits &&
             help.Resources == "search,users,socialgraph"
             select help)
            .SingleOrDefault();

        foreach (var category in helpResult.RateLimits)
        {
            Console.WriteLine("\nCategory: {0}", category.Key);

            foreach (var limit in category.Value)
            {
                Console.WriteLine(
                    "\n  Resource: {0}\n    Remaining: {1}\n    Reset: {2}\n    Limit: {3}",
                    limit.Resource, limit.Remaining, limit.Reset, limit.Limit);
            }
        }

            var tweets = from t in twitterCtx.Status
                         where t.Type == StatusType.User && t.ScreenName == screename && t.Count == 15
                         select t;

            if (tweets != null)
            {
                foreach (var tweetStatus in tweets)
                {
                    if (tweetStatus != null)
                    {
                        lstofTweets.Add(new TwitterProfileDetails { Name = tweetStatus.User.Name, ProfileImagePath = tweetStatus.User.ProfileImageUrl, Tweets = tweetStatus.Text, UserID = tweetStatus.User.Identifier.UserID, PostedDate = Convert.ToDateTime(tweetStatus.CreatedAt),ScreenName=screename });
                    }
                }
            }
            return lstofTweets;
    }

1. I am using above method has below..

       foreach (var screenObj in screenName)
       {
           var getTweets = api.GetAllTweets(api.AuthenticateCredentials, screenObj.UserName);
   
           foreach (var obj in getTweets)
           {
               using (DBcontext = new DBContext())
               {
                   tweets.Name = obj.Name;
                   tweets.ProfileImage = obj.ProfileImagePath;
                   tweets.PostedOn = obj.PostedDate;
                   tweets.Tweets = obj.Tweets;
                   tweets.CreatedOn = DateTime.Now;
                   tweets.ModifiedOn = DateTime.Now;
                   tweets.Status = EntityStatus.Active;
                   tweets.ScreenName = obj.ScreenName;
                   var exist = context.UserTweets.Any(user => user.Tweets.Equals(obj.Tweets));
                   if (!exist)
                       context.UserTweets.Add(tweets);
                   context.SaveChanges();
               }
           }
       }
   

Answer 1

I see that you found the Help/RateLimits query. There are various approaches you can take. e.g. add a delay between queries, delay the next query if the limit has been exceeded, or catch the exception and delay until the next 15 minute window.

If you want to monitor interactively, you can watch the rate limit for each query. The TwitterContext instance you use for performing the query contains RateLimitXxx properties that populate after every query. You'll need to read those values after the query, which appears to be inside your GetAllTweets method. You have to expose those values to your loop somehow, via return object, out params, static field, or whatever logic you feel is necessary.

// the first time through, you have the whole rate limit for the 15 minute window

foreach (var screenObj in screenName)
{
    var getTweets = api.GetAllTweets(api.AuthenticateCredentials, screenObj.UserName);

    // your processing logic ...

    // assuming you have the RateLimitXxx values in scope
    if (rateLimitRemaining == 0)
        Thread.Sleep(CalculateRemainingMilliseconds(RateLimitReset));


}

RateLimitRemaining is how many queries you can do in the current 15 minute window and RateLimitReset is the number of epoch seconds remaining until the rate limit resets (when you can start querying again).

It would be helpful to review the Twitter docs on Rate Limiting.

For reference, here are a couple other questions that might provide more ideas:

• Twitter rate limiting

• Get all followers using LINQ to Twitter