0

我正在使用具有 3 个节点的 RMQ 高可用集群,版本:3.8.3

规格:

  • 内存:4GB
  • CPU : 2CPU

间歇性地出现以下错误,并且一些节点因内存警报而崩溃。

Application mnesia exited with reason: stopped
wal: encountered error during recovery: badarg

完整的日志条目:

**********************************************************
*** Publishers will be blocked until this alarm clears ***
**********************************************************
2020-07-14 01:13:00.914 [warning] <0.328.0> rabbit_sysmon_handler busy_dist_port <0.456.0> [{name,rabbit_alarm},{initial_call,{gen_event,init_it,6}},{erlang,bif_return_trap,2},{message_queue_len,0}] {#Port<0.968>,unknown}
2020-07-14 01:13:02.838 [warning] <0.328.0> rabbit_sysmon_handler busy_dist_port <0.684.0> [{initial_call,{rabbit_prequeue,init,1}},{erts_internal,dsend_continue_trap,1},{message_queue_len,0}] {#Port<0.968>,unknown}
2020-07-14 01:31:34.457 [info] <0.8.0> Log file opened with Lager
2020-07-14 01:31:37.799 [info] <0.8.0> Feature flags: list of feature flags found:
2020-07-14 01:31:37.799 [info] <0.8.0> Feature flags:   [x] drop_unroutable_metric
2020-07-14 01:31:37.799 [info] <0.8.0> Feature flags:   [x] empty_basic_get_metric
2020-07-14 01:31:37.799 [info] <0.8.0> Feature flags:   [x] implicit_default_bindings
2020-07-14 01:31:37.799 [info] <0.8.0> Feature flags:   [x] quorum_queue
2020-07-14 01:31:37.800 [info] <0.8.0> Feature flags:   [x] virtual_host_metadata
2020-07-14 01:31:37.800 [info] <0.8.0> Feature flags: feature flag states written to disk: yes
2020-07-14 01:31:37.910 [info] <0.43.0> Application mnesia exited with reason: stopped
2020-07-14 01:31:38.072 [info] <0.395.0> ra: meta data store initialised. 0 record(s) recovered
2020-07-14 01:31:38.072 [info] <0.402.0> WAL: recovering ["/var/lib/rabbitmq/mnesia/rabbit@rmq-3/quorum/rabbit@rmq-3/00000058.wal"]
2020-07-14 01:31:38.518 [warning] <0.402.0> wal: encountered error during recovery: badarg

在这个时候我能够看到系统iowait很高,

在此处输入图像描述

而且我能够看到高 TCP 错误

在此处输入图像描述

这可能是什么原因?

任何帮助将不胜感激。

谢谢。

4

1 回答 1

0

这并不能解决节点崩溃问题,但是根据这个 Google groups postwal: encountered error during recovery: badarg可以忽略 3.8.3 中的消息:

这个错误信息完全没有影响,不会在3.8.4打印

所以也许这条线是一条红鲱鱼,你的问题在别处。

于 2020-10-05T15:09:58.840 回答