java - Tomcat 崩溃：Java 运行时环境内存不足，无法继续

Question

我们正在运行一个用户数量较少的 Web 应用程序，大约 7 个，但显示图像的负载很高。最近我们升级到 Java 6u43 和 Tomcat 6.036，在使用一段时间后（大约一周）tomcat 服务崩溃了。重新启动后，它可以运行几天，或者只运行几个小时，然后再次崩溃。即使在重新启动服务器后它再次崩溃，我们甚至在活动 10 分钟后看到它。

它总是在tomcat的bin文件夹中的hs_err_pid文件中带有相同的消息。请参阅下面的完整错误消息示例。总是本机内存不足，主活动线程始终是 JavaThread“C2 CompilerThread1”，但始终使用不同的方法。

我们在 Windows 2008 Service Pack 2、32 位、2 CPU、3GB 上运行。对于 tomcat xms 设置为 512KB 和 xmx 为 1536KB。没有其他具体设置。当它崩溃时，Java 堆远低于 1536 的完全使用量。

任何想法，建议这可能是什么？

（在 tomcat 升级之前，我们长期使用 tomcat 6.014 和 Java 6u02 没有任何问题）

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 629784 bytes for Chunk::new
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.cpp:317), pid=5156, tid=1820
#
# JRE version: 6.0_43-b01
# Java VM: Java HotSpot(TM) Server VM (20.14-b01 mixed mode windows-x86 )

---------------  T H R E A D  ---------------

Current thread (0x00f99800):  JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=1820, stack(0x66680000,0x666f0000)]

Stack: [0x66680000,0x666f0000],  sp=0x666ee694,  free space=441k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

[error occurred during error reporting (printing native stack), id 0xc0000005]


Current CompileTask:
C2:398266147 1679  !   org.apache.jasper.compiler.Compiler.isOutDated(Z)Z (485 bytes)


---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x66870c00 JavaThread "http-8080-4" daemon [_thread_blocked, id=4856, stack(0x6b6a0000,0x6b710000)]
  0x66870800 JavaThread "http-8080-3" daemon [_thread_blocked, id=5684, stack(0x6b170000,0x6b1e0000)]
  0x66870000 JavaThread "ajp-8018-5" daemon [_thread_blocked, id=3532, stack(0x6b100000,0x6b170000)]
  0x6686fc00 JavaThread "ajp-8018-4" daemon [_thread_blocked, id=1400, stack(0x6b050000,0x6b0c0000)]
  0x6686f400 JavaThread "ajp-8018-3" daemon [_thread_blocked, id=2804, stack(0x6afe0000,0x6b050000)]
  0x6686ec00 JavaThread "http-8080-2" daemon [_thread_blocked, id=3940, stack(0x68fd0000,0x69040000)]
  0x6686e800 JavaThread "http-8080-1" daemon [_thread_blocked, id=5168, stack(0x68f60000,0x68fd0000)]
  0x6686e000 JavaThread "ajp-8018-2" daemon [_thread_blocked, id=4260, stack(0x68ef0000,0x68f60000)]
  0x6686dc00 JavaThread "ajp-8018-1" daemon [_thread_blocked, id=5476, stack(0x68e80000,0x68ef0000)]
  0x6686d400 JavaThread "ajp-8018-Acceptor-0" daemon [_thread_in_native, id=2640, stack(0x68e10000,0x68e80000)]
  0x6686d000 JavaThread "ajp-8018-CometPoller-7" daemon [_thread_blocked, id=4672, stack(0x68da0000,0x68e10000)]
  0x6686c800 JavaThread "ajp-8018-CometPoller-6" daemon [_thread_blocked, id=4892, stack(0x68d30000,0x68da0000)]
  0x6686c000 JavaThread "ajp-8018-CometPoller-5" daemon [_thread_blocked, id=1836, stack(0x68cc0000,0x68d30000)]
  0x6686bc00 JavaThread "ajp-8018-CometPoller-4" daemon [_thread_blocked, id=5452, stack(0x68c50000,0x68cc0000)]
  0x6686b400 JavaThread "ajp-8018-CometPoller-3" daemon [_thread_blocked, id=584, stack(0x68be0000,0x68c50000)]
  0x66fc7000 JavaThread "ajp-8018-CometPoller-2" daemon [_thread_blocked, id=1932, stack(0x68b70000,0x68be0000)]
  0x66fc6800 JavaThread "ajp-8018-CometPoller-1" daemon [_thread_blocked, id=5540, stack(0x68b00000,0x68b70000)]
  0x66fc6000 JavaThread "ajp-8018-CometPoller-0" daemon [_thread_blocked, id=2728, stack(0x68a90000,0x68b00000)]
  0x66fc5c00 JavaThread "ajp-8018-Poller-7" daemon [_thread_blocked, id=5104, stack(0x68a20000,0x68a90000)]
  0x66fc5400 JavaThread "ajp-8018-Poller-6" daemon [_thread_blocked, id=3756, stack(0x689b0000,0x68a20000)]
  0x66fc5000 JavaThread "ajp-8018-Poller-5" daemon [_thread_blocked, id=4888, stack(0x68940000,0x689b0000)]
  0x66fc4800 JavaThread "ajp-8018-Poller-4" daemon [_thread_in_native, id=3924, stack(0x688d0000,0x68940000)]
  0x66fc4400 JavaThread "ajp-8018-Poller-3" daemon [_thread_blocked, id=2632, stack(0x68860000,0x688d0000)]
  0x66fc3c00 JavaThread "ajp-8018-Poller-2" daemon [_thread_blocked, id=3792, stack(0x687f0000,0x68860000)]
  0x66fc3400 JavaThread "ajp-8018-Poller-1" daemon [_thread_blocked, id=4840, stack(0x68380000,0x683f0000)]
  0x66fc3000 JavaThread "ajp-8018-Poller-0" daemon [_thread_blocked, id=5424, stack(0x68310000,0x68380000)]
  0x66fc2800 JavaThread "http-8080-Acceptor-0" daemon [_thread_in_native, id=5532, stack(0x682a0000,0x68310000)]
  0x66fc2400 JavaThread "http-8080-Sendfile-0" daemon [_thread_blocked, id=2560, stack(0x68230000,0x682a0000)]
  0x66fc1c00 JavaThread "http-8080-CometPoller-7" daemon [_thread_blocked, id=4264, stack(0x681c0000,0x68230000)]
  0x66fc1800 JavaThread "http-8080-CometPoller-6" daemon [_thread_blocked, id=5948, stack(0x68150000,0x681c0000)]
  0x66fc1000 JavaThread "http-8080-CometPoller-5" daemon [_thread_blocked, id=3768, stack(0x680e0000,0x68150000)]
  0x66fc0800 JavaThread "http-8080-CometPoller-4" daemon [_thread_blocked, id=5772, stack(0x68070000,0x680e0000)]
  0x66fc0400 JavaThread "http-8080-CometPoller-3" daemon [_thread_blocked, id=564, stack(0x68000000,0x68070000)]
  0x66fbfc00 JavaThread "http-8080-CometPoller-2" daemon [_thread_blocked, id=1192, stack(0x67f90000,0x68000000)]
  0x66fbf800 JavaThread "http-8080-CometPoller-1" daemon [_thread_blocked, id=2776, stack(0x67f20000,0x67f90000)]
  0x66c2f800 JavaThread "http-8080-CometPoller-0" daemon [_thread_blocked, id=1012, stack(0x67eb0000,0x67f20000)]
  0x66dbe400 JavaThread "http-8080-Poller-7" daemon [_thread_blocked, id=3288, stack(0x67e40000,0x67eb0000)]
  0x66d9d800 JavaThread "http-8080-Poller-6" daemon [_thread_blocked, id=3684, stack(0x67dd0000,0x67e40000)]
  0x669af000 JavaThread "http-8080-Poller-5" daemon [_thread_blocked, id=4244, stack(0x67d60000,0x67dd0000)]
  0x6701b000 JavaThread "http-8080-Poller-4" daemon [_thread_blocked, id=2604, stack(0x67a90000,0x67b00000)]
  0x66f4f000 JavaThread "http-8080-Poller-3" daemon [_thread_blocked, id=2276, stack(0x67a20000,0x67a90000)]
  0x66509000 JavaThread "http-8080-Poller-2" daemon [_thread_blocked, id=5324, stack(0x679b0000,0x67a20000)]
  0x66c5a400 JavaThread "http-8080-Poller-1" daemon [_thread_blocked, id=2628, stack(0x678f0000,0x67960000)]
  0x66b9a800 JavaThread "http-8080-Poller-0" daemon [_thread_blocked, id=5576, stack(0x67850000,0x678c0000)]
  0x668b4000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=3152, stack(0x677e0000,0x67850000)]
  0x66853000 JavaThread "GC Daemon" daemon [_thread_blocked, id=1636, stack(0x675e0000,0x67650000)]
  0x00f9c800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=3184, stack(0x666f0000,0x66760000)]
=>0x00f99800 JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=1820, stack(0x66680000,0x666f0000)]
  0x00f98400 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=3932, stack(0x66610000,0x66680000)]
  0x00f96c00 JavaThread "Attach Listener" daemon [_thread_blocked, id=2612, stack(0x663a0000,0x66410000)]
  0x00f95c00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=5128, stack(0x66330000,0x663a0000)]
  0x00f68800 JavaThread "Finalizer" daemon [_thread_blocked, id=4868, stack(0x662c0000,0x66330000)]
  0x00f62000 JavaThread "Reference Handler" daemon [_thread_blocked, id=5500, stack(0x66250000,0x662c0000)]
  0x00e5a000 JavaThread "main" [_thread_in_native, id=2800, stack(0x00d70000,0x00de0000)]

Other Threads:
  0x00f5e800 VMThread [stack: 0x661e0000,0x66250000] [id=4608]
  0x00f9d000 WatcherThread [stack: 0x66760000,0x667d0000] [id=3600]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap
 PSYoungGen      total 368192K, used 215481K [0x46860000, 0x65730000, 0x65c60000)
  eden space 230016K, 84% used [0x46860000,0x5263eae8,0x54900000)
  from space 138176K, 15% used [0x5cfb0000,0x5e43f968,0x656a0000)
  to   space 137920K, 0% used [0x54900000,0x54900000,0x5cfb0000)
 PSOldGen        total 349568K, used 320646K [0x08060000, 0x1d5c0000, 0x46860000)
  object space 349568K, 91% used [0x08060000,0x1b981b60,0x1d5c0000)
 PSPermGen       total 33664K, used 33565K [0x04060000, 0x06140000, 0x08060000)
  object space 33664K, 99% used [0x04060000,0x061277e8,0x06140000)

Code Cache  [0x00fa0000, 0x01570000, 0x03fa0000)
 total_blobs=1881 nmethods=1658 adapters=176 free_code_cache=44292736 largest_free_block=10112

Dynamic libraries:
0x00400000 - 0x00417000     D:\Kappa\apache-tomcat-6.0.36\bin\tomcat6.exe
0x77460000 - 0x77587000     C:\Windows\system32\ntdll.dll
0x76490000 - 0x7656c000     C:\Windows\system32\kernel32.dll
0x75ef0000 - 0x75fb6000     C:\Windows\system32\ADVAPI32.dll
0x75fc0000 - 0x76083000     C:\Windows\system32\RPCRT4.dll
0x76570000 - 0x77080000     C:\Windows\system32\SHELL32.dll
0x762d0000 - 0x7637a000     C:\Windows\system32\msvcrt.dll
0x75d30000 - 0x75d7b000     C:\Windows\system32\GDI32.dll
0x75e50000 - 0x75eed000     C:\Windows\system32\USER32.dll
0x76270000 - 0x762c9000     C:\Windows\system32\SHLWAPI.dll
0x775d0000 - 0x775ee000     C:\Windows\system32\IMM32.DLL
0x775f0000 - 0x776b8000     C:\Windows\system32\MSCTF.dll
0x77590000 - 0x77599000     C:\Windows\system32\LPK.DLL
0x75d80000 - 0x75dfd000     C:\Windows\system32\USP10.dll
0x748d0000 - 0x74a6e000     C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_6.0.6002.18005_none_5cb72f96088b0de0\comctl32.dll
0x6db90000 - 0x6dfea000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\server\jvm.dll
0x743e0000 - 0x74412000     C:\Windows\system32\WINMM.dll
0x77240000 - 0x77385000     C:\Windows\system32\ole32.dll
0x771b0000 - 0x7723d000     C:\Windows\system32\OLEAUT32.dll
0x74380000 - 0x743b9000     C:\Windows\system32\OLEACC.dll
0x7c340000 - 0x7c396000     C:\Windows\system32\MSVCR71.dll
0x75ae0000 - 0x75b0c000     C:\Windows\system32\apphelp.dll
0x6d870000 - 0x6d87c000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\verify.dll
0x6d3d0000 - 0x6d3ef000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\java.dll
0x6d8b0000 - 0x6d8bf000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\zip.dll
0x6d600000 - 0x6d609000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\management.dll
0x673d0000 - 0x674d7000     D:\Kappa\apache-tomcat-6.0.36\bin\tcnative-1.dll
0x75cd0000 - 0x75cd7000     C:\Windows\system32\PSAPI.DLL
0x775a0000 - 0x775cd000     C:\Windows\system32\WS2_32.dll
0x760a0000 - 0x760a6000     C:\Windows\system32\NSI.dll
0x75300000 - 0x7533b000     C:\Windows\system32\MSWSOCK.dll
0x75060000 - 0x7509b000     C:\Windows\system32\rsaenh.dll
0x6d850000 - 0x6d858000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\sunmscapi.dll
0x755e0000 - 0x756d2000     C:\Windows\system32\CRYPT32.dll
0x75740000 - 0x75752000     C:\Windows\system32\MSASN1.dll
0x75b90000 - 0x75bae000     C:\Windows\system32\USERENV.dll
0x75b70000 - 0x75b84000     C:\Windows\system32\Secur32.dll
0x74f00000 - 0x74f05000     C:\Windows\System32\wshtcpip.dll
0x6d6b0000 - 0x6d6c3000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\net.dll
0x75360000 - 0x75365000     C:\Windows\System32\wship6.dll
0x74c20000 - 0x74c2f000     C:\Windows\system32\NLAapi.dll
0x75540000 - 0x75559000     C:\Windows\system32\IPHLPAPI.DLL
0x75500000 - 0x75535000     C:\Windows\system32\dhcpcsvc.DLL
0x75780000 - 0x757ac000     C:\Windows\system32\DNSAPI.dll
0x754f0000 - 0x754f7000     C:\Windows\system32\WINNSI.DLL
0x754c0000 - 0x754e2000     C:\Windows\system32\dhcpcsvc6.DLL
0x74290000 - 0x7429f000     C:\Windows\system32\napinsp.dll
0x74270000 - 0x74278000     C:\Windows\System32\winrnr.dll
0x75e00000 - 0x75e49000     C:\Windows\system32\WLDAP32.dll
0x74280000 - 0x74286000     C:\Windows\system32\rasadhlp.dll
0x003d0000 - 0x003dd000     D:\Kappa\Java\jdk1.6.0_43\bin\ntlmauth.dll
0x75410000 - 0x75417000     C:\Windows\system32\credssp.dll
0x750d0000 - 0x75114000     C:\Windows\system32\schannel.dll
0x759a0000 - 0x75a16000     C:\Windows\system32\NETAPI32.dll
0x752c0000 - 0x752f7000     C:\Windows\system32\msv1_0.dll
0x757b0000 - 0x757c1000     C:\Windows\system32\cryptdll.dll
0x6d0b0000 - 0x6d1fc000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\awt.dll
0x707c0000 - 0x70802000     C:\Windows\system32\WINSPOOL.DRV
0x740d0000 - 0x74155000     C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.6001.18000_none_886786f450a74a05\COMCTL32.dll
0x00920000 - 0x0092b000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\mlib_jai_util.dll
0x6b250000 - 0x6b69e000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\mlib_jai_mmx.dll
0x6d220000 - 0x6d24f000     D:\Kappa\Java\jdk1.6.0_43\jre\bin\cmm.dll

VM Arguments:
jvm_args: -Dcatalina.base=D:\Kappa\apache-tomcat-6.0.36 -Dcatalina.home=D:\Kappa\apache-tomcat-6.0.36 -Djava.endorsed.dirs=D:\Kappa\apache-tomcat-6.0.36\endorsed -Djava.io.tmpdir=D:\Kappa\apache-tomcat-6.0.36\temp -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.util.logging.config.file=D:\Kappa\apache-tomcat-6.0.36\conf\logging.properties -Xms512m -Xmx1500m 
java_command: <unknown>
Launcher Type: generic

Environment Variables:
JAVA_HOME=D:\Kappa\Java\jdk1.6.0_43
CLASSPATH=D:\Kappa\apache-tomcat-6.0.36\lib\servlet-api.jar
PATH=C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;D:\Kappa\Java\jdk1.6.0_43\bin;C:\Program Files\Windows Imaging\
USERNAME=SVC-UPSLKappa-DIA
OS=Windows_NT
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 6, GenuineIntel



---------------  S Y S T E M  ---------------

OS: Windows Server 2008 Build 6002 Service Pack 2

CPU:total 2 (1 cores per cpu, 1 threads per core) family 6 model 23 stepping 6, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1

Memory: 4k page, physical 2095900k(292724k free), swap 4467300k(2557472k free)

vm_info: Java HotSpot(TM) Server VM (20.14-b01) for windows-x86 JRE (1.6.0_43-b01), built on Mar  1 2013 02:47:24 by "java_re" with MS VC++ 7.1 (VS2003)

time: Tue Oct 01 08:51:47 2013
elapsed time: 398205 seconds

Answer 1

进程堆！= java 堆。

在 32 位 Windows 上（除非您设置了一些引导开关）每个进程最大 2GB。这 2GB 必须支持：

• Java 堆（使用 Xms、Xmx 设置）
• 永久代
• 线程堆栈
• GC
• 代码缓存
• 本机 JVM 代码

可能有几件事我已经忘记了。

如果上述所有内容的总和超过 2GB，则进程崩溃。这就是你的情况发生的事情。

错误消息非常清楚地告诉您需要做什么。选择其中一个选项并实施。

Answer 2

使用像VisualVM这样的分析器。连接它以监控您的 Tomcat 服务器的内存使用情况（远程配置）。它会持续监控并显示你的堆分配。您还可以看到内存中保存的对象。

如果发生这种情况，您要么达到了进程的内存限制（您将在视觉 vm 中看到总使用量）。它甚至可能是 JVM 中的内存泄漏或错误。检查您是否使用的是 64 位版本。但首先使用 VisualVM（或类似的）检查对象创建和堆。您可以通过图表判断您的消费是否以不同的方式上升。

您甚至可以比较使用以前使用的 Java 版本获得的堆使用模式。通过这种方式，您可以将旧工作设置的内存消耗与新的有问题的设置进行比较。

Answer 3

在我们的例子中，它发生是因为内存使用率甚至在启动 tomcat 之前就超过了 95%。解决方案是稍微释放 RAM，以便 tomcat 能够运行该项目的最低限度。