Best approach for web service that calls other web services

Question

The scenario is:

1. client makes request to server A
2. Server A makes potentially multiple requests to server B. Edit to clarify, server A makes the requests concurrently using Futures.
3. Server A blocks until all results return
4. Server A collates responses into single response and returns to client

So currently the approach is the block using java Futures and return the results when they come in. My concern is this is quite thread intensive and I'm worried we will run out of resources on the server. I was thinking of going async in server A by using DeferredResult but since there is a thread for each request to server B, that doesn't seem to carry much benefit.

Alternatively, we could do

1. Server A makes multiple requests to Server B
2. Server B immediate returns a token for each request
3. Server A tracks the tokens and returns DeferredResult
4. Server B gets each result, puts it in memcache, sends a JMS message to server A. Edit: I just realized I could send the result in the JMS message, so no need for memcache at all.
5. Server A can complete the DeferredResult after it collects all the responses from JMS.

This has the advantage of being much much simpler in code (no complicated futures work) at the cost of being more complicated architecturally. It also introduces more layers (JMS) for failure. It also has the advantage of being much less resource intensive.

This is a high use system. Calls to system B can take anywhere from 1 to several seconds.

Answer 1

I think your original plan sounds better than the 'alternative' you described. KISS. Your current approach is very simple.

Structurally what you are doing has no need to be thread intensive (no more than one thread to service the call). You can invoke many calls to Service B just by invoking async tasks. Maybe that's already how the Java Future<> implementation works (not sure - would have to look). other stack overflow article appears to indicate it works that way. Otherwise, you can use another java mechanism to invoke async web requests.

If your web service engine in server A is written to specially handle DeferredResult objects (I doubt this but you can check) - then your plan to use DeferredResult would work well. But most likely, your webservice just will force a GET call on the result of the websevice call, and will block the webservice call handling thread while the work completes whereever it comes from.

You did hint at the possibility of another strategy. Depending on the nature of your calls from ServerA to ServerB, perhaps those could be sped up or cached. Using something like redis/memchached (or just an in memory cache inside your ServerA) MIGHT be an effective way to make your overall throughput much faster, but that depends ENTIRELY on the cachability of calls from ServerA to ServerB (which you said nothing about). If you want to experiement with this caching approach, use a HashMap, and some abstraction to count hits/misses, to see how effective the approach is, before trying to scale up to something more complex like memcached.