lucene - 在 WhitespaceAnalyzer 中使用停用词

Question

Lucene 的 StandardAnalyzer 在索引时从字符串/首字母缩略词中删除点。我希望 Lucene 保留点，因此我使用的是 WhitespaceAnalyzer 类。

我可以将我的停用词列表提供给 StandardAnalyzer...但是我如何将它提供给 WhitespaceAnalyzer？

谢谢阅读。

Answer 1

Create your own analyzer by extending WhiteSpaceAnalyzer and override tokenStream method as follows.

public TokenStream tokenStream(String fieldName, Reader reader) {
    TokenStream result = super.tokenStream(fieldName, reader);
    result = new StopFilter(result, stopSet);
    return result;
}

Here the stopSet is the Set of stop words, which you could get by adding a constructor to your analyzer which accepts a list of stop words.

You may also wish to override reusableTokenStream() method in similar fashion if you plan to reuse the TokenStream.