2

我正在使用 Elasticsearch 6.4 和 Kibana 6。我也在使用 Sentil 插件。

https://github.com/sirensolutions/sentinl

这个插件是 xpact 观察者和监控的免费替代品。但是，我在正确编写观察者查询时遇到了一些困难。我只想拉出最新的百分比值，并在设置百分比高于 90% 时发出警报。

我的查询：

    {
  "actions": {
    "email_html_alarm_4b1479be-5e70-492e-9e02-fb08412510ee": {
      "name": "Check CPU Usage Usage for ip-172-0-0-0",
      "throttle_period": "1m",
      "email_html": {
        "stateless": false,
        "to": "g@g.com",
        "from": "g@g.com",
        "subject": "Critical CPU Usage Percent over 90% : {{ payload.aggregations.cpu_used.value  }}",
        "priority": "high",
        "html": "<p>Your elasticsearch is using more than 90% of its CPU: {{ payload.aggregations.cpu_used.value }}. Please scale the cluster. found by the watcher <i>{{watcher.title}}</i>.</p>"
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "index": [
          "metricbeat-*"
        ],
        "body": {
          "from": 0,
          "size": 1,
          "query": {
            "bool": {
              "must": [
                {
                  "exists": {
                    "field": "system.cpu.total.pct"
                  }
                }
              ],
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-20s"
                    }
                  }
                },
                {
                  "query_string": {
                    "query": "beat.name:ip-172-0-0-0"
                  }
                }
              ]
            }
          },
          "aggs": {
            "cpu_used": {
              "terms": {
                "field": "system.cpu.total.pct",
                "size": 1
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "array_compare": {
      "payload.aggregations.cpu_used.buckets": {
        "path": "key",
        "gt": {
          "value": 0.9
        }
      }
    }
  },
  "trigger": {
    "schedule": {
      "later": "every 2 minutes"
    }
  },
  "disable": true,
  "report": true,
  "title": "CPU Check for ip-172-0-0-0",
  "wizard": {},
  "save_payload": false,
  "spy": false,
  "impersonate": false
}

这个观察者的问题是它会在值读取 0.1445555 并且它不大于或等于 gte 所代表的 0.9000 时触发。

当我在 kibana 的开发人员工具中运行查询时，它返回以下内容：

{
  "took": 86,
  "timed_out": false,
  "_shards": {
    "total": 354,
    "successful": 354,
    "skipped": 331,
    "failed": 0
  },
  "hits": {
    "total": 2,
    "max_score": 1,
    "hits": [
      {
        "_index": "metricbeat-production-6.4.2-2018.10.25",
        "_type": "doc",
        "_id": "T4KBrGYBIGbF3wm-Wgpd",
        "_score": 1,
        "_source": {
          "@timestamp": "2018-10-25T18:34:09.872Z",
          "host": {
            "name": "ip-172-0-0-0"
          },
          "metricset": {
            "name": "cpu",
            "module": "system",
            "rtt": 153
          },
          "system": {
            "cpu": {
              "user": {
                "pct": 0.1235
              },
              "idle": {
                "pct": 1.4769
              },
              "nice": {
                "pct": 0
              },
              "irq": {
                "pct": 0
              },
              "steal": {
                "pct": 0.002
              },
              "total": {
                "pct": 0.1406
              },
              "cores": 2,
              "softirq": {
                "pct": 0.001
              },
              "system": {
                "pct": 0.0141
              },
              "iowait": {
                "pct": 0.3825
              }
            }
          },
          "beat": {
            "hostname": "ip-172-0-0-0",
            "version": "6.4.2",
            "name": "ip-172-0-0-0"
          }
        }
      }
    ]
  },
  "aggregations": {
    "cpu_used": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 1,
      "buckets": [
        {
          "key": 0.14100000000000001,
          "doc_count": 1
        }
      ]
    }
  }
}

鉴于数组比较逻辑，这不应该正确触发警报，因为键值只有 .14 而不是 gte .90，所以这让我相信我没有得到正确的值。

Sentil 将数组比较描述为：

数组比较条件 使用array_compare 比较一个值数组。例如，如果聚合中至少有一个存储桶的 doc_count 大于或等于 25，则以下 array_compare 条件返回 true：

"condition": {
  "array_compare": {
    "payload.aggregations.top_amounts.buckets" : { 
      "path": "doc_count" ,
      "gte": { 
        "value": 25, 
      }
    }
  }
}

选项

名称 说明 array.path 执行上下文中数组的路径，以点表示法指定 array.path.path 要评估的每个数组元素中的字段的路径 array.path.operator.quantifier 需要多少个匹配项比较评估为真：some或all。默认为some，必须至少有一个匹配项。如果数组为空，则比较结果为 false array.path.operator.value 要比较的值

有人可以帮助我解决我的观察者和/或查询做错了什么。我似乎无法获得百分比并检查百分比值。

4

1 回答 1

1

这些是最终使用指标节拍的查询。

{
  "actions": {
    "email_html_alarm_4b1479be": {
      "name": "Check Disk Usage for ip-0-0-0-0",
      "throttle_period": "1m",
      "email_html": {
        "stateless": false,
        "to": "test@test.com",
        "from": "test@test.com",
        "subject": "Critical /dev/xvda1 Available Bytes: {{ payload.aggregations.disk_used.value  }}",
        "priority": "high",
        "html": "<p>Your elasticsearch node only has: {{ payload.aggregations.disk_used.value }} bytes available. Please snapshot and clean old indexes. found by the watcher <i>{{watcher.title}}</i>.</p>"
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "index": [
          "metricbeat-*"
        ],
        "body": {
          "from": 0,
          "size": 1,
          "query": {
            "bool": {
              "must": [
                {
                  "exists": {
                    "field": "system.filesystem.used.pct"
                  }
                },
                {
                  "match": {
                    "system.filesystem.device_name": "/dev/xvda1"
                  }
                }
              ],
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-1m"
                    }
                  }
                },
                {
                  "query_string": {
                    "query": "beat.name:ip-0-0-0-0"
                  }
                }
              ]
            }
          },
          "aggs": {
            "disk_used": {
              "avg": {
                "field": "system.filesystem.available"
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "script": {
      "script": "payload.aggregations.disk_used.value < 490497080832"
    }
  },
  "trigger": {
    "schedule": {
      "later": "every 2 minutes"
    }
  },
  "disable": false,
  "report": true,
  "title": "ESDisk",
  "wizard": {},
  "save_payload": false,
  "spy": false,
  "impersonate": false
}



######CPUCHECK

第二次查询

{
  "actions": {
    "email_html_alarm_4b1479be": {
      "name": "Check CPU Usage Usage for ip-0-0-0-0",
      "throttle_period": "1m",
      "email_html": {
        "stateless": false,
        "to": "test@test.com",
        "from": "test@test.com",
        "subject": "Critical CPU Usage Percent over 90% : {{ payload.aggregations.cpu_used.value  }}",
        "priority": "high",
        "html": "<p>Your elasticsearch is using more than 90% of its CPU: {{ payload.aggregations.cpu_used.value }}. Please scale the cluster. found by the watcher <i>{{watcher.title}}</i>.</p>"
      }
    }
  },
  "input": {
    "search": {
      "request": {
        "index": [
          "metricbeat-*"
        ],
        "body": {
          "from": 0,
          "size": 1,
          "query": {
            "bool": {
              "must": [
                {
                  "exists": {
                    "field": "system.cpu.total.pct"
                  }
                }
              ],
              "filter": [
                {
                  "range": {
                    "@timestamp": {
                      "gte": "now-20s"
                    }
                  }
                },
                {
                  "query_string": {
                    "query": "beat.name:ip-0-0-0-0"
                  }
                }
              ]
            }
          },
          "aggs": {
            "cpu_used": {
              "terms": {
                "field": "system.cpu.total.pct",
                "size": 1
              }
            }
          }
        }
      }
    }
  },
  "condition": {
    "array_compare": {
      "payload.aggregations.cpu_used.buckets": {
        "path": "key",
        "gte": {
          "value": 0.90
        }
      }
    }
  },
  "trigger": {
    "schedule": {
      "later": "every 2 minutes"
    }
  },
  "disable": true,
  "report": true,
  "title": "CPU Check for ip-0-0-0-0",
  "wizard": {},
  "save_payload": false,
  "spy": false,
  "impersonate": false
}
于 2019-07-07T20:13:12.357 回答