我有两个 J2EE 应用程序 A1 和 A2 部署在同一个应用程序服务器 IBM Websphere 7.0 上。两个应用程序都部署在同一个配置文件上。每个应用程序都有自己的 Web GUI 并提供多种 Web 服务，例如 A1 提供 S1，A2 提供 S2。
在正常工作过程中，几个用户正在使用 GUI，而另一个外部系统正在调用 Web 服务 S1 和 S2。
Web 服务 S1 在他的工作中也称为 Web 服务 S2。
有时，每周大约有一次应用程序服务器挂起。用户无法进入 GUI，其他系统无法访问 Web 服务。
线程转储显示在从套接字读取期间许多线程正在等待。所有这些线程都是本地 Web 服务调用。同时，Web 容器池中大约有 5 个空闲线程。总 CPU 负载小于 10%。内存使用也OK。垃圾收集器没有运行，当 GC 运行时大约需要 200 毫秒，所以 GC 不是挂起的原因。
所以我的问题是：
- 为什么本地 Web 服务调用挂起？
- 当 Web 容器池中有几个空闲线程时，为什么用户无法访问 Web GUI？
=====
Web 容器线程：
WebContainer : 0 Runnable 3836 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 1 Runnable 3840 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 10 Runnable 1352 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 11 Waiting on condition 1736 java/lang/StackTraceElement.appendTo(StackTraceElement.java:180(Compiled Code)) 74
WebContainer : 12 Waiting on condition 3444 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 13 Runnable 3056 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 14 Runnable 3508 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 15 Runnable 3344 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 16 Runnable 1592 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 17 Waiting on condition 1744 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 18 Waiting on condition 2108 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 19 Runnable 684 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 2 Runnable 1552 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 20 Runnable 4824 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 21 Runnable 4132 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 22 Runnable 4116 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 23 Runnable 4164 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 24 Runnable 2668 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 25 Runnable 4212 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 26 Runnable 4020 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 27 Runnable 496 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 28 Runnable 2448 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 29 Runnable 3428 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 3 Runnable 3908 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 30 Runnable 4168 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 31 Runnable 4160 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 32 Runnable 3328 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 33 Runnable 3348 java/net/SocketInputStream.socketRead0(Native Method) 69
WebContainer : 34 Runnable 3324 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 35 Runnable 4308 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 36 Blocked 3164 java/lang/Object.wait(Native Method) 6
WebContainer : 37 Blocked 852 java/lang/Object.wait(Native Method) 6
WebContainer : 38 Waiting on condition 4288 java/lang/Object.wait(Native Method) 6
WebContainer : 39 Blocked 5056 java/lang/Object.wait(Native Method) 6
WebContainer : 4 Runnable 2400 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 5 Runnable 4092 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 6 Runnable 3968 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 7 Runnable 1832 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 8 Waiting on condition 3356 java/net/SocketInputStream.socketRead0(Native Method) 68
WebContainer : 9 Runnable 2236 java/net/SocketInputStream.socketRead0(Native Method) 68
=====
线程 36-39 是免费的：
at java/lang/Object.wait(Native Method)
at java/lang/Object.wait(Object.java:196(Compiled Code))
at com/ibm/ws/util/BoundedBuffer.waitGet_(BoundedBuffer.java:187(Compiled Code))
at com/ibm/ws/util/BoundedBuffer.take(BoundedBuffer.java:549(Compiled Code))
at com/ibm/ws/util/ThreadPool.getTask(ThreadPool.java:899(Compiled Code))
at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1653(Compiled Code))
=====
几乎所有的剩余线程都有相同的堆栈跟踪：
at java/net/SocketInputStream.socketRead0(Native Method)
at java/net/SocketInputStream.read(SocketInputStream.java:140(Compiled Code))
at java/io/BufferedInputStream.fill(BufferedInputStream.java:229(Compiled Code))
at java/io/BufferedInputStream.read(BufferedInputStream.java:248(Compiled Code))
at org/apache/axis/transport/http/HTTPSender.readHeadersFromSocket(HTTPSender.java:583(Compiled Code))
at org/apache/axis/transport/http/HTTPSender.invoke(HTTPSender.java:143(Compiled Code))
at org/apache/axis/strategies/InvocationStrategy.visit(InvocationStrategy.java:32(Compiled Code))
at org/apache/axis/SimpleChain.doVisiting(SimpleChain.java:118(Compiled Code))
at org/apache/axis/SimpleChain.invoke(SimpleChain.java:83(Compiled Code))
at org/apache/axis/client/AxisClient.invoke(AxisClient.java:165(Compiled Code))
at org/apache/axis/client/Call.invokeEngine(Call.java:2784(Compiled Code))
at org/apache/axis/client/Call.invoke(Call.java:2767(Compiled Code))
at org/apache/axis/client/Call.invoke(Call.java:2443(Compiled Code))
at org/apache/axis/client/Call.invoke(Call.java:2366(Compiled Code))
at org/apache/axis/client/Call.invoke(Call.java:1812(Compiled Code))
call web service S2 on the same host here
...
My web service code here.
...
at sun/reflect/GeneratedMethodAccessor366.invoke(Bytecode PC:124(Compiled Code))
at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37(Compiled Code))
at java/lang/reflect/Method.invoke(Method.java:611(Compiled Code))
at org/apache/axis/providers/java/RPCProvider.invokeMethod(RPCProvider.java:397(Compiled Code))
at org/apache/axis/providers/java/EJBProvider.invokeMethod(EJBProvider.java:459(Compiled Code))
at org/apache/axis/providers/java/RPCProvider.processMessage(RPCProvider.java:186(Compiled Code))
at org/apache/axis/providers/java/JavaProvider.invoke(JavaProvider.java:323(Compiled Code))
at org/apache/axis/strategies/InvocationStrategy.visit(InvocationStrategy.java:32(Compiled Code))
at org/apache/axis/SimpleChain.doVisiting(SimpleChain.java:118(Compiled Code))
at org/apache/axis/SimpleChain.invoke(SimpleChain.java:83(Compiled Code))
at org/apache/axis/handlers/soap/SOAPService.invoke(SOAPService.java:454(Compiled Code))
at org/apache/axis/server/AxisServer.invoke(AxisServer.java:281(Compiled Code))
at org/apache/axis/transport/http/AxisServlet.doPost(AxisServlet.java:699(Compiled Code))
at javax/servlet/http/HttpServlet.service(HttpServlet.java:738(Compiled Code))
at org/apache/axis/transport/http/AxisServletBase.service(AxisServletBase.java:327(Compiled Code))
at javax/servlet/http/HttpServlet.service(HttpServlet.java:831(Compiled Code))
at com/ibm/ws/webcontainer/servlet/ServletWrapper.service(ServletWrapper.java:1657(Compiled Code))
at com/ibm/ws/webcontainer/servlet/ServletWrapper.service(ServletWrapper.java:1597(Compiled Code))
at com/ibm/ws/webcontainer/filter/WebAppFilterChain.doFilter(WebAppFilterChain.java:131(Compiled Code))
at com/ibm/ws/webcontainer/filter/FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:188(Compiled Code))
at com/ibm/ws/webcontainer/filter/WebAppFilterChain.doFilter(WebAppFilterChain.java:116(Compiled Code))
at com/ibm/ws/webcontainer/filter/WebAppFilterChain._doFilter(WebAppFilterChain.java:77(Compiled Code))
at com/ibm/ws/webcontainer/filter/WebAppFilterManager.doFilter(WebAppFilterManager.java:908(Compiled Code))
at com/ibm/ws/webcontainer/servlet/ServletWrapper.handleRequest(ServletWrapper.java:934(Compiled Code))
at com/ibm/ws/webcontainer/servlet/ServletWrapper.handleRequest(ServletWrapper.java:502(Compiled Code))
at com/ibm/ws/webcontainer/servlet/ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181(Compiled Code))
at com/ibm/ws/webcontainer/servlet/CacheServletWrapper.handleRequest(CacheServletWrapper.java:91(Compiled Code))
at com/ibm/ws/webcontainer/WebContainer.handleRequest(WebContainer.java:864(Compiled Code))
at com/ibm/ws/webcontainer/WSWebContainer.handleRequest(WSWebContainer.java:1592(Compiled Code))
at com/ibm/ws/webcontainer/channel/WCChannelLink.ready(WCChannelLink.java:186(Compiled Code))
at com/ibm/ws/http/channel/inbound/impl/HttpInboundLink.handleDiscrimination(HttpInboundLink.java:452(Compiled Code))
at com/ibm/ws/http/channel/inbound/impl/HttpInboundLink.handleNewRequest(HttpInboundLink.java:511(Compiled Code))
at com/ibm/ws/http/channel/inbound/impl/HttpInboundLink.processRequest(HttpInboundLink.java:305(Compiled Code))
at com/ibm/ws/http/channel/inbound/impl/HttpInboundLink.ready(HttpInboundLink.java:276(Compiled Code))
at com/ibm/ws/tcp/channel/impl/NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214(Compiled Code))
at com/ibm/ws/tcp/channel/impl/NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113(Compiled Code))
at com/ibm/ws/tcp/channel/impl/AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165(Compiled Code))
at com/ibm/io/async/AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217(Compiled Code))
at com/ibm/io/async/AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161(Compiled Code))
at com/ibm/io/async/AsyncFuture.completed(AsyncFuture.java:138(Compiled Code))
at com/ibm/io/async/ResultHandler.complete(ResultHandler.java:204(Compiled Code))
at com/ibm/io/async/ResultHandler.runEventProcessingLoop(ResultHandler.java:775(Compiled Code))
at com/ibm/io/async/ResultHandler$2.run(ResultHandler.java:905(Compiled Code))
at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1646(Compiled Code))
谢谢。
斯坦