1

StandaloneAhcWSClient如何在 Scala中发出大量简单的 get 请求？（它是与 Play2 框架捆绑的默认 http 客户端）。就我而言，我收到了大约 100K GET 请求来向外部 API 发出请求。Future.traverse()不削减它，有没有更好的方法，也许是一种像流一样处理url列表的方法？

这是我现在拥有的代码：https ://scastie.scala-lang.org/HgrIyR23TmG12j3MzMCxUw

它最多可以在列表中使用一定数量的 url，但会中断大量的 URL，但有一个例外java.lang.IllegalStateException: failed to create a child event loop

4

1 回答 1

1

这就是我最终得到的结果：

import play.api.libs.json.JsValue
import akka.actor.ActorSystem

import scala.concurrent.{Await, Future}
import scala.concurrent.duration.Duration
import scala.concurrent.ExecutionContext.Implicits.global
import play.api.libs.ws.StandaloneWSClient
import akka.stream.ActorMaterializer
import play.api.libs.ws.ahc.StandaloneAhcWSClient
import play.api.libs.ws.JsonBodyReadables._
import play.api.libs.json._

implicit val system: ActorSystem = ActorSystem()
system.registerOnTermination {
  System.exit(0)
}
implicit val materializer: ActorMaterializer = ActorMaterializer()

def getAllRecApiResponses(urls: List[String])(
  implicit actorSystem: ActorSystem,
  materializer: ActorMaterializer): List[JsValue] = {
  implicit val wsClient: StandaloneWSClient = StandaloneAhcWSClient()

  val res: Future[List[JsValue]] = Future.traverse(urls)(urlString => {
    wsClient.url(urlString).get().map(_.body[JsValue]).recover {
      case ex: Exception => {
        println( s"Url call returned exception for url $urlString: $ex" )
        JsNull
      }
    }
  }) andThen { case _ => wsClient.close() }

  Await.result(res, Duration.Inf)
}

val result = getAllRecApiResponses(List.fill(10)("https://jsonplaceholder.typicode.com/todos/1"))

result foreach println

使用以下 build.sbt：

scalaVersion := "2.11.12"

val liftVersion = "2.6"

libraryDependencies ++= Seq(
  "com.typesafe.play" %% "play-ahc-ws-standalone" % "2.0.3",
  "com.typesafe.play" %% "play-ws-standalone-json" % "2.0.3",
  "com.typesafe.play" %% "play-json" % "2.7.2"
)
于 2019-04-17T18:46:55.547 回答