Consume REST API from Python: do I need an async library?

Question

I have a REST API and now I want to create a web site that will use this API as only and primary datasource. The system is distributed: REST API is on one group of machines and the site will be on the other(s).

I'm expecting to have quite a lot of load, so I'd like to make requests as efficient, as possible.

Do I need some async HTTP requests library or any HTTP client library will work?

API is done using Flask, web site will be also built using Flask and Jinja as template engine.

Answer 1

You could use gevent with Flask to get asynchronous I/O from normally synchronous libraries. See this question for an example of someone getting help with doing that.

You could also run Flask behind gunicorn, which has support for spawning multiple workers (threads, processes, or greenlets) for handling concurrent requests. If you were to take that approach, Flask would remain completely synchronous, and gunicorn would handle creating multiple Flask instances to handle concurrent requests.

Answer 2

Start simple and use the way, which seems to be easy to use for you. Consider optimization to be done later on only if needed.

Use of async libraries would come into play as helpful if you would have thousands of request a second. Much sooner you are likely to have performance problems related to database (if you use it), which is not to be resolved by async magic.

Answer 3

Actually your API is on a separate machine. Even if you make your client calls asynchronous , it will not have any impact on the server. By making your calls asynchronous, your thread in the client will not wait for the response. Your server will react the same when call is sync /async.

And if you want to make your calls async , please check http://stackandqueue.com/?p=57 . It uses unirest to make both get and post async calls