Question

In my Play! 2.1 Rest API App I have installed New Relic.

All of my controller Actions inherit from a method that adds a timeout to the future of the response. If any such method takes longer than 20 seconds, the request is terminated and the result is a 5XX Error.

The code is basically this:

val timeout = 20

action(request).orTimeout(
     name + " backend timed-out after "+timeout+" seconds", timeout * 1000).map { 
     resultOrTimeout => { //... process response or timeout with fold

The problem I'm having is that when analyzing the data in new relic, Im getting that average response times of 20 seconds always.

When looking at the trace, I can see that new relic interprets the timeout function as the container of the response.

Slowest components                         Count    Duration    %
Async Wait                                  7       20,000 ms   100%
Action$$anonfun$apply$1.apply()             2       2 ms         0%
PlayDefaultUpstreamHandler$$an....apply()     1       1 ms         0%
PlayDefaultUpstream....$$anonfun$24.apply() 1      1 ms         0%
SmaugController$class.akkify()               1       0 ms         0%
PlayDefaultUpstreamHandler.handleAction$1() 1       0 ms          0%
Total                                               20,000 ms   100%

Is there any way I can prevent new-relic from considering that timeout?

Thanks!

EDIT: I expanded the transaction to get more information:

Duration (ms)   Duration (%)    Segment Drilldown   Timestamp
20,000  100.00%    HttpRequestDecoder.unfoldAndFireMessageReceived()
20,000  100.00%    Async Wait
Stack trace

scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:23)
      java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1146)

     java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615)

                                   java.lang.Thread.run (Thread.java:679)

107  0.53%   SmaugController$class.akkify() 

As you can see, the real work is being done in the akkify method, which takes 107 ms, all the rest is being consumed by the Async Wait call

Was it helpful?

Solution

Unfortunately, it is currently not possible to ignore that particular timeout in New Relic.

However, the 3.4.1 release of the New Relic Java Agent supports the handle-timeouts sample code documented in Play 2.2.1: http://www.playframework.com/documentation/2.2.1/ScalaAsync

You can download it here: https://download.newrelic.com/newrelic/java-agent/newrelic-agent/3.4.1/

import play.api.libs.concurrent.Execution.Implicits.defaultContext
import scala.concurrent.duration._

def index = Action.async {
  val futureInt = scala.concurrent.Future { intensiveComputation() }
  val timeoutFuture = play.api.libs.concurrent.Promise.timeout("Oops", 1.second)
  Future.firstCompletedOf(Seq(futureInt, timeoutFuture)).map {
    case i: Int => Ok("Got result: " + i)
    case t: String => InternalServerError(t)
  }
}
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top