I've actually somehow misread the documentation. Enumerator.fromStream has an implicit ExecutionContext you can provide.
If you create a dedicated context for this specific type of operations, you can still experience thread starvation, but you're in control of which threadpool gets that problem.
We're using play! so we can just configure akka threadpools in our application.conf:
# this is a root value in the application.conf, but you can put it anywhere
# as long as you provide the full path to the .lookup() function
my-contexts {
s3-streaming {
fork-join-executor {
parallelism-min = 50
parallelism-max = 50
}
}
}
and use them in the code like this:
object MyContexts {
val s3Streaming: ExecutionContext =
Akka.system.dispatchers.lookup("my-contexts.s3-streaming")
}
...
Enumerator.fromStream(stream)(MyContexts.s3Streaming)