I'm trying to create an analyzer that would remove (or replace by white/empty space) a quoted sentence within a document.
Such as: this is my \"test document\"
I'd like, for example, the term vector to be: [this, is, my]
I'm trying to create an analyzer that would remove (or replace by white/empty space) a quoted sentence within a document.
Such as: this is my \"test document\"
I'd like, for example, the term vector to be: [this, is, my]
Daniel Answer 是正确的,但由于缺少相应的正则表达式和替换,我提供了它,其中包括对您的文本的测试。
使用模式替换字符的索引设置如下。
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"char_filter": [
"my_char_filter"
],
"filter": [
"lowercase"
]
}
},
"char_filter": {
"my_char_filter": {
"type": "pattern_replace",
"pattern": "\"(.*?)\"",
"replacement": ""
}
}
}
}
}
之后使用分析 API生成以下标记:
POST _analyze
{
"text": "this is my \"test document\"",
"analyzer" : "my_analyzer"
}
上述 API 的输出:
{
"tokens": [
{
"token": "this",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "is",
"start_offset": 5,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "my",
"start_offset": 8,
"end_offset": 10,
"type": "<ALPHANUM>",
"position": 2
}
]
}
您可以使用此字段的模式替换字符过滤器配置您自己的分析器,该过滤器将转义双引号之间的所有内容替换为空。