python - Ngram 模型错误。需要计算困惑度

Question

我收到一个 NGramModel 错误...请告诉我哪里可能出错了。

Traceback (most recent call last):
  File "ai7.py", line 87, in <module>
    tt=NgramModel(1, tText, estimator)
NameError: name 'NgramModel' is not defined

我的目标是计算文本的困惑度。

f_in = open("science.txt", 'r');
ln = f_in.read()    

words = nltk.word_tokenize(ln)
my_bigrams = nltk.bigrams(words)
my_trigrams = nltk.trigrams(words)

s=""
tText = Text(words)
tText1 = Text(my_bigrams)
tText2 = Text(my_trigrams)
estimator = lambda fdist, bins: LidstoneProbDist(fdist, 0.2)


tt=NgramModel(1, tText, estimator)
tt1=NgramModel(2, tText1, estimator)
tt2=NgramModel(3, tText2, estimator)


print tt.perplexity(tText)
print tt1.perplexity(tText1)
print tt2.perplexity(tText2)

Answer 1

您忘记导入NgramModel

您应该执行以下操作：

from nltk.model.ngram import NgramModel