python - 有效地搜索文档中的术语列表

Question

我有数千个同义词的列表。我还有数以万计的文档要搜索这些术语。使用python（或伪代码）什么是这样做的有效方法？

# this would work for single word synonyms, but there are multiple word synonyms too
synonymSet = set([...])
wordsInDocument = set([...])
synonymsInDocument = synonymSet.intersection(wordsInDocument)

# this would work, but sounds slow
matches = []
for document in documents:
    for synonym in synonymSet:
        if synonym in document:
            matches.append(synonym)

这个问题有没有好的解决方案，还是需要一段时间？先感谢您

Answer 1

如何从同义词列表中构建正则表达式：

import re

pattern = "|".join(synonymList)
regex = re.compile(pattern)

matches = regex.findall(document) # get a list of the matched synonyms
matchedSynonyms = set(matches)    # eliminate duplicates using a set