mongodb - 使用 DMS 从 mongodb 加载数据到 elasticsearch 时出错，有什么想法吗？

Question

我正在尝试使用 AWS DMS 并将数据从 mongodb 传输到 amazon elasticsearch。

我在 CloudWatch 中遇到以下日志。

{
   "error": {
       "root_cause": [
           {
               "type": "mapper_parsing_exception",
               "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
           }
       ],
       "type": "mapper_parsing_exception",
       "reason": "Field [_id] is a metadata field and cannot be added inside a document. Use the index API request parameters."
   },
   "status": 400
}

这是我对 mongo db 源的配置。它_id as a separete column启用了复选框。我尝试禁用它，它说没有主键。你们知道有什么可以解决的吗？

快速说明：我已经添加了 _id 字段的映射，old_id现在它不会导入所有其他字段，即使我将它们添加到映射中

Answer 1

As ElasticSearch will not support the LOB data type, Other fields are not migrated.

Add additional transformation rule to change the data type to String

{
      "rule-type": "transformation",
      "rule-id": "3",
      "rule-name": "3",
      "rule-action": "change-data-type",
      "rule-target": "column",
      "object-locator": {
        "schema-name": "test",
        "table-name": "%",
        "column-name": "%"
      },
      "data-type": {
        "type": "string",
        "length": "30"
      }
}