1

我正在尝试从 gensim 实现 doc2vec，但有一些错误，并且网络上没有足够的文档或帮助。这是我的工作代码的一部分：

from gensim.models import Doc2Vec
from gensim.models.doc2vec import LabeledSentence

class LabeledLineSentence(object):
    def __init__(self, filename):
        self.filename = filename
    def __iter__(self):
        with open(self.filename, 'r') as f:
            for uid, line in enumerate(f):
                print LabeledSentence(line.split(), tags=['TXT_%s' % uid])
                yield LabeledSentence(words=line.split(), tags=['TXT_%s' % uid])

sentences = LabeledLineSentence('myfile.txt')

我的 txt 文件是什么样的：

  1 hi how are you
  2 hi how are you
  3 hi how are you
  4 its such a great day
  5 its such a great day
  6 its such a great day
  7 i like dogs
  8 i like cats
  9 i like snakes
 10 the ice cream was yummy
 11 the cake was awesome

初始化模型

model = Doc2Vec(alpha=0.025, min_alpha=0.025, size=50, window=5, min_count=5,
                dm=1, workers=8, sample=1e-5)

示例打印输出：

LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_0'])
LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_1'])
LabeledSentence(['hi', 'how', 'are', 'you'], ['TXT_2'])
LabeledSentence(['its', 'such', 'a', 'great', 'day'], ['TXT_3'])
LabeledSentence(['its', 'such', 'a', 'great', 'day'], ['TXT_4'])

这是错误所在：

for epoch in range(500):
    try:
        print 'epoch %d' % (epoch)
        model.train(sentences)
        model.alpha *= 0.99
        model.min_alpha = model.alpha
    except (KeyboardInterrupt, SystemExit):
        break

RuntimeError: you must first build vocabulary before training the model

知道为什么吗？

4

1 回答 1

1

调用 model.build_vocab 将修复错误。

请参阅本教程https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/doc2vec-lee.ipynb

于 2016-10-30T10:08:23.873 回答