Fastest way to scrape followers from a Twitter account with Node.js [closed]

Question

Closed. This question is opinion-based. It is not currently accepting answers.

---

Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.

Closed 7 years ago.

Improve this question

Due to the limit rate of twitter API I would like to know how could I scrape the number of followers from a Twitter account with Node.js in a very very fast way? Do you think Node.js is the best solution to do it?

I have no idea of how it works but I thought that perhaps scraping to the Twitter mobile version ( example https://mobile.twitter.com/android) would be faster than the computer version.

What do you think? How could I do it?

Thanks!!

Answer 1

Now that the Twitter v1.0 API has been deprecated, you must use oauth signed requests. Which means Twitter knows when you abuse the system.

Docs on oath signed requests: https://dev.twitter.com/docs/auth/oauth

Docs on rate limiting: https://dev.twitter.com/docs/rate-limiting/1.1

Rate limiting applies to the API, not unauthenticated http requests which simply scrape the content.

If you just want the follower count and not the individual follower ids / details, you can simply scrape the page using request and use jQuery to filter down the results using selectors.

eg. https://twitter.com/foofighters

You can run this client side to preview the results:

$('.js-mini-profile-stats a.js-nav[data-nav=followers] strong').attr('title')