2

I have a spray http client which is running in a server X, which will make connections to server Y. Server Y is kind of slow(will take 3+ sec for a request)

This is my http client code invocation:

def get() {
    val result = for {
       response <- IO(Http).ask(HttpRequest(GET,Uri(getUri(msg)),headers)).mapTo[HttpResponse]
    } yield response

    result onComplete {
      case Success(res) => sendSuccess(res)
      case Failure(error) => sendError(res)
    }
}

These are the configurations I have in application.conf:

spray.can {
    client {
            request-timeout = 30s
            response-chunk-aggregation-limit = 0
            max-connections = 50
            warn-on-illegal-headers = off
        }
        host-connector {
            max-connections = 128
            idle-timeout = 3s
          }
    }

Now I tried to abuse the server X with large number of concurrent requests(using ab with n=1000 and c=100).

Till 900 requests it went fine. After that the server threw lot of exceptions and I couldn't hit the server after that. These are the exceptions:

[info] [ERROR] [03/28/2015 17:33:13.276] [squbs-akka.actor.default-dispatcher-6] [akka://squbs/system/IO-TCP/selectors/$a/0] Accept error: could not accept new connection

[info] java.io.IOException: Too many open files [info] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) [info] at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241) [info] at akka.io.TcpListener.acceptAllPending(TcpListener.scala:103)

and on further hitting the same server, it threw the below exception:

[info] [ERROR] [03/28/2015 17:53:16.735] [hcp-client-akka.actor.default-dispatcher-6] [akka://hcp-client/system/IO-TCP/selectors] null [info] akka.actor.ActorInitializationException: exception during creation

[info] at akka.actor.ActorInitializationException$.apply(Actor.scala:164)

[info] at akka.actor.ActorCell.create(ActorCell.scala:596)

[info] Caused by: java.lang.reflect.InvocationTargetException

[info] at sun.reflect.GeneratedConstructorAccessor59.newInstance(Unknown Source)

[info] Caused by: java.io.IOException: Too many open files [info] at sun.nio.ch.IOUtil.makePipe(Native Method)

I was previously using apache http client(which was synchronous) which was able to handle 10000+ requests with concurrency of 100.

I'm not sure I'm missing something. Any help would be appreciated.

4

1 回答 1

3

问题是每次调用get()方法时，它都会创建一个新的参与者，该参与者至少创建一个到远程服务器的连接。此外，您永远不会关闭该参与者，因此每个此类连接都会离开，直到超时。

您只需要一个这样的参与者来管理您的所有 HTTP 请求，从而将其从方法IO(Http)中取出get()并仅调用一次。重用返回ActorRef的所有请求到该服务器。在应用程序关闭时将其关闭。

例如：

val system: ActorSystem = ...
val io = IO(Http)(system)
io ! Http.Bind( ...

def get(): Unit = {
  ...
  io.ask ...
  // or
  io.tell ...
}
于 2015-03-29T07:05:36.483 回答