Using Thread.sleep() inside an foreach in scala

Question

I've a list of URLs inside a List.

I want to get the data by calling WS.url(currurl).get(). However, I want add a delay between each request. Can I add Thread.sleep() ? or is there another way of doing this?

 one.foreach {
    currurl => {

  import play.api.libs.ws.WS
  println("using " + currurl)
  val p = WS.url(currurl).get()
  p.onComplete {
    case Success(s) => {
         //do something 
    }
    case Failure(f) => {
      println("failed")
    }
  }
}

}

Answer 1

Sure, you can call Thread.sleep inside your foreach function, and it will do what you expect.

That will tie up a thread, though. If this is just some utility that you need to run sometimes, then who cares, but if it's part of some server you are trying to write and you might tie up many threads, then you probably want to do better. One way you could do better is to use Akka (it looks like you are using Play, so you are already using Akka) to implement the delay -- write an actor that uses scheduler.schedule to arrange to receive a message periodically, and then handle one request each time the message is read. Note that Akka's scheduler itself ties up a thread, but it can then send periodic messages to an arbitrary number of actors.

Answer 2

You can do it with scalaz-stream

  import org.joda.time.format.DateTimeFormat
  import scala.concurrent.duration._
  import scalaz.stream._
  import scalaz.stream.io._
  import scalaz.concurrent.Task

  type URL = String
  type Fetched = String

  val format = DateTimeFormat.mediumTime()

  val urls: Seq[URL] =
    "http://google.com" :: "http://amazon.com" :: "http://yahoo.com" :: Nil

  val fetchUrl = channel[URL, Fetched] {
    url => Task.delay(s"Fetched " +
      s"url:$url " +
      s"at: ${format.print(System.currentTimeMillis())}")
  }

  val P = Process
  val process =
    (P.awakeEvery(1.second) zipWith P.emitAll(urls))((b, url) => url).
      through(fetchUrl)

  val fetched = process.runLog.run
  fetched.foreach(println)

Output:

Fetched url:http://google.com at: 1:04:25 PM
Fetched url:http://amazon.com at: 1:04:26 PM
Fetched url:http://yahoo.com at: 1:04:27 PM