0

我创建了一个摄取管道，如下所示将字段拆分为单词：

POST _ingest/pipeline/_simulate
{
    "pipeline": {
        "description": "String cutting processing",
        "processors": [
            {
                "split": {
                    "field": "foo",
                    "separator": "|"
                }
            }
        ]
    },
    "docs": [
        {
            "_source": {
                "foo": "apple|time"
            }
        }
    ]
}

但它将字段拆分为字符：

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "foo" : [
            "a",
            "p",
            "p",
            "l",
            "e",
            "|",
            "t",
            "i",
            "m",
            "e"
          ]
        }
      }
    }
  ]
}

如果我用逗号替换分隔符，相同的管道将字段拆分为单词：

POST _ingest/pipeline/_simulate
{
    "pipeline": {
        "description": "String cutting processing",
        "processors": [
            {
                "split": {
                    "field": "foo",
                    "separator": ","
                }
            }
        ]
    },
    "docs": [
        {
            "_source": {
                "foo": "apple,time"
            }
        }
    ]
}

那么输出将是：

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "foo" : [
            "apple",
            "time"
          ]
        }
      }
    }
  ]
}

当分隔符为“|”时，如何将字段拆分为单词？我的下一个问题是如何将此摄取管道应用于现有索引？我尝试了这个解决方案，但它对我不起作用。

编辑

这是包含将两个部分分配给两列的文档的整个管道：

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "description": """combined fields are text that contain  "|" to separate two fields""",
    "processors": [
      {
        "split": {
          "field": "dv_m",
          "separator": "|",
          "target_field": "dv_m_splited"
        }
      },
      {
        "set": {
          "field": "dv_metric_prod",
          "value": "{{dv_m_splited.1}}",
          "override": false
        }
      },
      {
        "set": {
          "field": "dv_metric_section",
          "value": "{{dv_m_splited.2}}",
          "override": false
        }
      }
    ]
  },
  "docs": [
    {

      "_source": {

        "dv_m": "amaze_inc|Understanding"

      }
    }
  ]
}

这会产生以下响应：

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "dv_metric_prod" : "m",
          "dv_m_splited" : [
            "a",
            "m",
            "a",
            "z",
            "e",
            "_",
            "i",
            "n",
            "c",
            "|",
            "U",
            "n",
            "d",
            "e",
            "r",
            "s",
            "t",
            "a",
            "n",
            "d",
            "i",
            "n",
            "g"
          ],
          "dv_metric_section" : "a",
          "dv_m" : "amaze_inc|Understanding"
        },
        "_ingest" : {
          "timestamp" : "2021-08-02T08:33:58.2234143Z"
        }
      }
    }
  ]
}

如果我设置"separator": "\\|"，那么我会得到这个错误：

{
  "docs" : [
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "general_script_exception",
            "reason" : "Error running com.github.mustachejava.codes.DefaultMustache@776f8239"
          }
        ],
        "type" : "general_script_exception",
        "reason" : "Error running com.github.mustachejava.codes.DefaultMustache@776f8239",
        "caused_by" : {
          "type" : "mustache_exception",
          "reason" : "Failed to get value for dv_m_splited.2 @[query-template:1]",
          "caused_by" : {
            "type" : "mustache_exception",
            "reason" : "2 @[query-template:1]",
            "caused_by" : {
              "type" : "index_out_of_bounds_exception",
              "reason" : "2"
            }
          }
        }
      }
    }
  ]
}
4

1 回答 1

1

解决方案相当简单：只需转义您的分隔符。

由于拆分处理器中的separator字段是正则表达式，因此您需要对特殊字符进行转义，例如.|

您还需要逃脱它两次。

所以你的代码只缺少双重转义部分：

POST _ingest/pipeline/_simulate

{
    "pipeline": {
        "description": "String cutting processing",
        "processors": [
            {
                "split": {
                    "field": "foo",
                    "separator": "\\|"
                }
            }
        ]
    },
    "docs": [
        {
            "_source": {
                "foo": "apple|time"
            }
        }
    ]
}

更新

您没有提到或者我错过了您想要将值分配给两个单独字段的部分。

在这种情况下，您应该使用dissect而不是split. 它更短、更简单、更干净。请参阅此处的文档。

POST _ingest/pipeline/_simulate

{
  "pipeline": {
    "description": """combined fields are text that contain  "|" to separate two fields""",
    "processors": [
      {
        "dissect": {
          "field": "dv_m",
          "pattern": "%{dv_metric_prod}|%{dv_metric_section}"
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "dv_m": "amaze_inc|Understanding"
      }
    }
  ]
}

结果

{
  "docs" : [
    {
      "doc" : {
        "_index" : "_index",
        "_type" : "_doc",
        "_id" : "_id",
        "_source" : {
          "dv_metric_prod" : "amaze_inc",
          "dv_metric_section" : "Understanding",
          "dv_m" : "amaze_inc|Understanding"
        },
        "_ingest" : {
          "timestamp" : "2021-08-18T07:39:12.84910326Z"
        }
      }
    }
  ]
}

附录

如果使用split而不是dissect

您的数组索引错误。没有{{dv_m_splited.2}}像数组索引从 0 开始这样的事情，并且您只有两个结果。

这是使用split处理器时正确的管道。

POST _ingest/pipeline/_simulate

{
  "pipeline": {
    "description": """combined fields are text that contain  "|" to separate two fields""",
    "processors": [
      {
        "split": {
          "field": "dv_m",
          "separator": "\\|",
          "target_field": "dv_m_splited"
        }
      },
      {
        "set": {
          "field": "dv_metric_prod",
          "value": "{{dv_m_splited.0}}",
          "override": false
        }
      },
      {
        "set": {
          "field": "dv_metric_section",
          "value": "{{dv_m_splited.1}}",
          "override": false
        }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "dv_m": "amaze_inc|Understanding"
      }
    }
  ]
}
于 2021-07-30T11:56:38.157 回答