3

与弹性搜索聚合作斗争 - 可能需要一些建议......

elasticsearch 版本：版本：1.4.1，内部版本：89d3241/2014-11-26T15:49:29Z，JVM：1.7.0_72

样本数据集：

{
  "_index": "logstash-2014.12.17",
  "_type": "netflow",
  "_id": "AUpaDdUVUcM5Us_C6x7Z",
  "_score": 1,
  "_source": {
    "message": "<27>Dec 17 22:01:02 es01 nfcapd[29441]: expip=10.245.132.16 fweventtime=2014-12-17 22:01:02.793 fwevent=DENIED srcip=78.110.142.76 dstip=179.24.227.252 srcport=62327 dstport=41863 proto=UDP input=3 output=4 inbytes=0 outbytes=0 postnatsrcip=78.110.142.76 postnatdstip=179.24.227.252 postnatsrcport=62327 postnatdstport=41863 ingressacl=0x45b0635e/0x9872d678/0x724bf9a4 egressacl=0x0/0x0/0x0",
    "@version": "1",
    "@timestamp": "2014-12-17T21:01:02.794Z",
    "type": "netflow",
    "host": "127.0.0.1",
    "timestamp": "Dec 17 22:01:02",
    "hostname": "es01",
    "expip": "10.245.132.16",
    "time": "2014-12-17 22:01:02.793",
    "fwevent": "DENIED",
    "srcip": "78.110.142.76",
    "dstip": "179.24.227.252",
    "srcport": "62327",
    "dstport": "41863",
    "proto": "UDP",
    "output": "4",
    "inbytes": "0",
    "outbytes": "0",
    "postnatsrcip": "78.110.142.76",
    "postnatdstip": "179.24.227.252",
    "postnatsrcport": "62327",
    "postnatdstport": "41863",
    "ingressacl1": "0x45b0635e",
    "ingressacl2": "0x9872d678",
    "ingressacl3": "0x724bf9a4",
    "egressacl1": "0x0",
    "egressacl2": "0x0",
    "egressacl3": "0x0",
    "srcgeo": {
      "country_code3": "CHE",
      "latitude": 47,
      "longitude": 8,
      "location": [
        8,
        47
      ]
    }
  }
}

示例查询：

GET _search
{
  "size": 1,
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "@timestamp": {
            "gt": "2014-12-17T21:00:00"
          }
        }
      }
    }
  },
  "aggs": {
    "proto": {
      "terms": {
        "field": "proto"
      },
      "aggs": {
        "traffic_sum": {
          "sum": {
            "field": "outbytes"
          }
        }
      }
    }
  }
}

导致错误：

{
  "error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; 
            shardFailures {[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][0]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][1]:
            RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
              nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][2]:
            RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
              nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][3]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][4]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]",
"status": 500
}

* 只使用一个聚合就可以正常工作 - 如果我插入第二个聚合会失败 * 有什么想法吗？

4

1 回答 1

2

这是重要的部分：

ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData 
cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]"

您正在尝试使用字符串字段进行求和。

这个领域是问题：

"outbytes": "0",

任何一个

  1. 删除现有数据并通过发布包含"outbytes": 0（注意缺少引号）的文档来创建数字字段类型。
  2. 删除现有数据并创建一个明确的映射，并将字段outbytes设置为数字。
  3. 保留您的数据，但更新聚合以调用执行字符串到数字转换的脚本。

我的建议是选择选项 2。

于 2014-12-18T08:26:11.160 回答