问题行为
- MongoDB 在生产环境中经常崩溃。每天至少一次。在每次崩溃之前,日志中只会出现如下内容:“缓慢的 WT 事务”。
崩溃前的可疑日志 [grep]:
2018-12-05T16:59:02.569+0000 D STORAGE [conn1170] Slow WT transaction. Lifetime of SnapshotId 1356150 was 154ms
2018-12-05T16:59:38.214+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1357284 was 154ms
2018-12-05T16:59:38.390+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1357291 was 103ms
2018-12-05T17:00:03.076+0000 D STORAGE [conn1170] Slow WT transaction. Lifetime of SnapshotId 1357925 was 158ms
2018-12-05T17:00:24.940+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1358710 was 115ms
2018-12-05T17:00:46.096+0000 D STORAGE [conn8] Slow WT transaction. Lifetime of SnapshotId 1359381 was 105ms
2018-12-05T17:00:59.304+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1359903 was 142ms
2018-12-05T17:01:13.309+0000 D STORAGE [conn26] Slow WT transaction. Lifetime of SnapshotId 1360311 was 164ms
2018-12-05T17:01:13.554+0000 D STORAGE [conn26] Slow WT transaction. Lifetime of SnapshotId 1360362 was 143ms
2018-12-05T17:01:23.541+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1360671 was 116ms
2018-12-05T17:01:23.761+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1360679 was 110ms
2018-12-05T17:01:29.340+0000 D STORAGE [conn4] Slow WT transaction. Lifetime of SnapshotId 1360880 was 203ms
环境:操作系统+浏览器/客户端+版本
MongoDB 版本信息:
db version v3.6.8
git version: 6bc9ed599c3fa164703346a22bad17e33fa913e4
OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
allocator: tcmalloc
modules: none
build environment:
distmod: rhel70
distarch: x86_64
target_arch: x86_64
操作系统信息:
NAME=CentOS Linux
VERSION=7 (Core)
ID=centos
ID_LIKE: rhel fedora
VERSION_ID=7
PRETTY_NAME=CentOS Linux 7 (Core)
下面是配置文件-
mongod.conf
# for documentation of all options, see:
# http://docs.mongodb.org/manual/reference/configuration-options/
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
# Where and how to store data.
storage:
dbPath: /apps/mongodb/data
journal:
enabled: true
# engine:
# mmapv1:
# wiredTiger
# how the process runs
processManagement:
fork: true # fork and run in background
pidFilePath: /var/run/mongodb/mongod.pid
timeZoneInfo: /usr/share/zoneinfo
# network interfaces
net:
port: 27017
bindIp: 127.0.0.1 # Listen to local interface only, comment to listen on all interfaces.
#security:
#operationProfiling:
#replication:
#sharding:
## Enterprise-Only Options
#auditLog:
#snmp:
可能的解决方案 [已尝试]
- 尝试增加系统的 ulimits 但在 2 天后连续发出。
当前超限:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 64045
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 128000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 64045
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
如果您需要更多信息,请告诉我。