1

我有运行tomcat(9.0.39.0)服务器,由用户cpappt启动并使用pid:1682650运行。该服务器运行的机器不是裸机,而是一VMware台机器,JVM来自OpenJ9

> ps -ef | grep Bootstrap
cpappt   1682650       1  0 Jan01 ?        01:09:58 /srv/jdk11/bin/java
  -Djava.util.logging.config.file=/srv/test/cpappt/tomcat/conf/logging.properties
  -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -server 
  -Xms32m -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError
  -Dcrewportalenvironment=test
  -Denvironment=test
  -Doracle.net.tns_admin=/CONFIG/global/ORA/HA
  -Dspring.profiles.active=test,notification-services-intern
  -Djdk.tls.ephemeralDHKeySize=2048
  -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
  -Dorg.apache.catalina.security.SecurityListener.UMASK=0022
  -Djava.awt.headless=true
  -Dcom.sun.management.jmxremote
  -Dfile.encoding=UTF-8
  -Dsun.jnu.encoding=UTF-8
  -Dcom.sun.management.jmxremote.port=7305
  -Dcom.sun.management.jmxremote.ssl=false
  -Dcom.sun.management.jmxremote.password.file=conf/jmxremote.password
  -Dignore.endorsed.dirs= -classpath /srv/test/cpappt/tomcat/bin/bootstrap.jar:/srv/test/cpappt/tomcat/bin/tomcat-juli.jar
  -Dcatalina.base=/srv/test/cpappt/tomcat
  -Dcatalina.home=/srv/test/cpappt/tomcat
  -Djava.io.tmpdir=/srv/test/cpappt/tomcat/temp org.apache.catalina.startup.Bootstrap start

我想提取一个heap dumptomcat 来分析潜在的内存泄漏。

我尝试使用jps获取pidtomcat 服务器的,不幸的是我只看到了pidjps本身。

> jps -l
952152 jdk.jcmd/openj9.tools.attach.diagnostics.tools.Jps

并且我也以jps启动 tomcat 服务器的同一用户身份运行该命令

> whoami 
cpappt

我也试过运行jps命令,java.io.tmpdir在tomcat的启动脚本中添加指定的,结果和之前一样(注:tmpdir本身为空)。

jps -l -J-Djava.io.tmpdir=/srv/test/cpappt/tomcat/temp

(补充说明:如果我启动了一个小的“Foo 程序”,jps就是向我展示了pid这个程序。)

然后我尝试使用失败的命令创建堆转储:pidps

> jmap -dump:live,format=b,file=/tmp/dump.hprof 1682650
unrecognized option -dump:live,format=b,file=/tmp/dump.hprof
jmap: obtain heap information about a Java process
 Usage:
    jmap <option>* <vmid>
        <vmid>: Attach API VM ID as shown in jps or other Attach API-based tools
        <vmid>s are read from stdin if none are supplied as arguments
    -histo: print statistics about classes on the heap, including number of objects and aggregate size
    -histo:live : Print only live objects
    -J: supply arguments to the Java VM running jmap
NOTE: this utility might significantly affect the performance of the target VM.
At least one option must be selected.

如果我只是尝试查看直方图,这也会失败:

> jmap -histo:live 1682650
Error getting data from 1682650: Exception connecting to 1682650

jcmd无法创建转储:

> jcmd 1682650 GC.heap_dump /tmp/heapdump
Error getting data from 1682650: Exception connecting to 1682650
...

我不知何故怀疑这两个问题 (jpsjmap) 是相关的......所以,如果有人知道可能是什么问题,甚至我可以如何创建堆转储,那就太好了。

来自 tomcat 服务器(和 java)的附加信息:

Server version number: 9.0.39.0
OS Name:               Linux
OS Version:            5.7.0-0.bpo.2-amd64
Architektur:           amd64
Java Home:             /srv/jdk-11.0.8+10_openj9
JVM Version:           11.0.8+10
JVM Hersteller:        Eclipse OpenJ9

> java -version
openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.8+10)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.21.0, JRE 11 Linux amd64-64-Bit Compressed References 20200715_697 (JIT enabled, AOT enabled)
OpenJ9   - 34cf4c075
OMR      - 113e54219
JCL      - 95bb504fbb based on jdk-11.0.8+10)

> which java
/srv/jdk11/bin/java

> which jps
/srv/jdk11/bin/java

> which jmap 
/srv/jdk11/bin/jmap

更新:2021-01-09

我终于可以找到知道 jmx 用户密码的同事,所以使用jconsole我能够触发 a heap dump,但我只能调用triggerClassicHeapDump(). 我将文件复制到我的本地机器并想用Eclipse MemoryAnalyzer(我还添加了Diagnostic Tool Framework for Java (DTFJ))来分析它。

但是当我尝试打开文件时,我现在收到一条错误消息......

Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'. Check the error log for further details.
Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'. Check the error log for further details.
Error opening heap dump 'heapdump.20210119.100934.2621412.0001.txt'
Unable to read dump .../heapdump.20210119.100934.2621412.0001.phd metafile .../heapdump.20210119.100934.2621412.0001.txt in DTFJ format DTFJ-PHD (java.io.IOException)
Unable to read dump .../heapdump.20210119.100934.2621412.0001.phd metafile .../heapdump.20210119.100934.2621412.0001.txt in DTFJ format DTFJ-PHD
Error parsing Javacore (java.io.IOException)
Error parsing Javacore
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.parser.ParserException)
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException)
Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Unable to read dump .../heapdump.20210119.100934.2621412.0001.txt metafile null in DTFJ format DTFJ-Javacore (java.io.IOException)
Unable to read dump .../heapdump.20210119.100934.2621412.0001.txt metafile null in DTFJ format DTFJ-Javacore
Error parsing Javacore (java.io.IOException)
Error parsing Javacore
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.parser.ParserException)
com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException: Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
Maximum line length (32768) exceeded. Input file corrupt or not a javacore. (com.ibm.dtfj.javacore.parser.framework.scanner.ScannerException)
Maximum line length (32768) exceeded. Input file corrupt or not a javacore.
4

2 回答 2

1

我已将-Xdump:heap:events=user(请参阅Xdump Option Builder)添加到 tomcat 的启动脚本并重新启动服务器。

> ps -ef | grep Bootstrap
cpappt   1919301       1 99 14:38 pts/0    00:01:00 /srv/jdk11/bin/java 
 -Xms32m -Xmx2048m -XX:+HeapDumpOnOutOfMemoryError
 -Xdump:heap:events=user
 ...

kill -3 <pid>现在我可以使用(看看超级用户:what-does-kill-3-mean )创建一个堆转储。在目录中创建了两个文件catalina.base

  • javacore.<XXX>.txt(原始线程转储(-3 := SIGQUIT))和
  • heapdump.<YYY>.phd(嗯,实际的堆转储)

最后我能够打开heapdump.<YYY>.phd文件Eclipse MAT

补充说明:我们将openj9版本更新为当前版本(23.0)。

> java -version 
openjdk version "11.0.9" 2020-10-20
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9+11)
Eclipse OpenJ9 VM AdoptOpenJDK (build openj9-0.23.0, JRE 11 Linux amd64-64-Bit Compressed References 20201022_810 (JIT enabled, AOT enabled)

现在jps也确实找到了tomcat服务器

> jps -l
1919301 org.apache.catalina.startup.Bootstrap
1921897 jdk.jcmd/openj9.tools.attach.diagnostics.tools.Jps

jcmd有效:

> jcmd 1919301 Dump.heap dump
Dump written to .../tomcat/dump
于 2021-01-19T13:57:59.890 回答
0

还可以考虑使用 -Xdump:system:events=user,因为生成的核心文件包含更多信息 - 但是 IBM DTFJ 读取 OpenJ9 生成的核心文件存在问题。

Memory Analyzer 还可以生成堆转储 - 请参阅文件 > 获取堆转储,但“附加 API”和“使用 Helper JVM 附加 API”可能需要一些配置。Memory Analyzer 1.12 在获取堆转储方面有一些改进,因此请尝试最新版本。

于 2021-06-23T08:51:36.853 回答