The links will be processed in parallel when you map extractData
over the collection. Consider a slightly simplified example:
import scala.concurrent._
import ExecutionContext.Implicits.global
def extractData(s: String) = future {
printf("Starting: %s\n", s)
val i = s.toInt
printf("Done: %s\n", s)
i
}
val xs = (0 to 5).map(_.toString).toList
val parsed = Future.sequence(xs map extractData)
Now you'll see something like the following, which makes it clear that these things aren't being processed sequentially:
Starting: 0
Done: 0
Starting: 2
Done: 2
Starting: 1
Starting: 4
Done: 1
Starting: 3
Starting: 5
Done: 5
Done: 4
Done: 3
Note that you can use Future.traverse
to avoid creating the intermediate list of futures:
val parsed = Future.traverse(xs)(extractData)
In either case you can block with Await
:
val res = Await.result(parsed, duration.Duration.Inf)
As a footnote: I don't know if you're planning to use Dispatch to perform the HTTP requests, but if not, it's worth a look. It also provides nicely integrated JSON parsing, and the documentation is full of useful examples of how to work with futures.