0

我有一个使用 SOLR 数据库的 PHP 应用程序。当我执行 /terms 请求时出现问题(terms doc

所以我们感兴趣的文件部分是

poi: "Bistriţa",
...
text: [
"ddt",
"Numeric",
"/14/Gagaga 2/11/Economics/17/datenow",
"/20/Daniel_same/11/Economics/17/datenow",
"0/Gagaga 2",
"1/Gagaga 2/Economics",
"2/Gagaga 2/Economics/datenow",
"0/Daniel_same",
"1/Daniel_same/Economics",
"2/Daniel_same/Economics/datenow",
"ppla",
"seat of a first-order administrative division",
"/19/Daniel_same/1071/Plurinational State of Bolivia/2269/Cuba/2272/Bistriţa",
"0/Daniel_same",
"1/Daniel_same/Plurinational State of Bolivia",
"2/Daniel_same/Plurinational State of Bolivia/Cuba",
"3/Daniel_same/Plurinational State of Bolivia/Cuba/Bistriţa",
"0/Undefined_activity",
"Year",
"0/1999",
"0/1999",
"Measured",
"",
"utf8"
],

请求是

http://localhost:8080/solr/terms
?wt=json
&indent=true
&terms.sort=count
&terms.mincount=1
&terms.limit=10
&terms.regex.flag=case_insensitive
&terms.regex=.*bi.*
&terms.fl=text

回应是

{
    responseHeader: {
        status: 0,
        QTime: 4
    },
    terms: {
        text: [
            "bistriå",
            16
        ]
    }
}

结果的问题是结果文本被截断。我期待“BistriÅ£a”,它是城市Bistrița的 UTF-8 编码。但结果似乎在特殊字符处被截断。

奇怪的是,如果我使用字段名称“poi”而不是“text”进行请求,我会得到正确的响应

http://localhost:8080/solr/terms
?wt=json
&indent=true
&terms.sort=count
&terms.mincount=1
&terms.limit=10
&terms.regex.flag=case_insensitive
&terms.regex=.*bi.*
&terms.fl=poi

{
    responseHeader: {
        status: 0,
        QTime: 4
    },
    terms: {
        text: [
            "Bistriţa",
            16
        ]
    }
}

所以这个词不会被截断。

这两个字段之间的最大区别是类型。Poi有一个字符串类型,而text有一个text_general类型。text_general 类型在这样的模式中定义

<fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <!-- in this example, we will only use synonyms at query time
    <filter class="solr.SynonymFilterFactory" synonyms="index_synonyms.txt" ignoreCase="true" expand="false"/>
    -->
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
  <analyzer type="query">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
    <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
    <filter class="solr.LowerCaseFilterFactory"/>
  </analyzer>
</fieldType>

如果被问到,我可以提供更多细节。不知道我现在可以添加什么,而不是让问题过于膨胀。

4

1 回答 1

0

You probably want to consider using the AsciiFoldingFilterFactory in your text_general field to appropriately handle the special character. Additionally, please refer to the Language Analysis support provided by Solr that may be of use to you.

于 2013-05-08T17:29:41.397 回答