Parallelization/threads inside a web request in Scala/Java?

Question

I have a webservice that is doing a number of database queries (to MongoDB) and then doing a non-trivial amount of processing on the records returned by those database queries. Each web request needs to run about 15-45 of these requests which is causing unacceptable page response times as each query + processing takes about 20ms. Caching will not work as the cache-miss rate will be high and data needs to be as fresh as possible.

I had planned on using Scala + Akka and actors to parallelize the class executing and processing the queries but I know creating threads inside a servlet container is discouraged. What is the recommended way of doing threading/parallelization inside a web request in Scala?

Answer 1

I don't see why you can't use Akka's Futures for this (I'm using the 2.0 SNAPSHOT).

import akka.dispatch.Future
Future.sequence(1 to 45 map { i =>
  Future {
    ... make database query i ...
  }
}) onComplete { future =>
  future.result.map { resultList =>
    // resultList.foldLeft...
    // resultList.map...
    // resultList.foreach...
    // resultList.whatever...
  }
}

(And there's certainly more than one way to deal with Futures in Akka to suit your needs. That example may not be ideal for what you want)

Have a look at the excellent Akka documentation for more info: Akka 1.2R6 Future documentation

As I mentioned in the comments above, I've used the Akka HTTP module to build highly scalable and reliable apps in the past, and they're all Actor / Future based.