1

我一直在研究潜在语义分析（lsa）并应用了这个例子：https ://radimrehurek.com/gensim/tut2.html

它包括在主题下聚类的术语，但找不到任何我们如何在主题下聚类文档。

在那个例子中，它说“看起来，根据 LSI，“trees”、“graph”和“minors”都是相关的词（并且对第一个主题的方向贡献最大），而第二个主题实际上关注本身与所有其他词。正如预期的那样，前五个文档与第二个主题的相关性更强，而其余四个文档与第一个主题相关。

我们如何将这五个文档与 Python 代码关联到相关主题？

你可以在下面找到我的 python 代码。我将不胜感激任何帮助。

from numpy import asarray
from gensim import corpora, models, similarities

#https://radimrehurek.com/gensim/tut2.html
documents = ["Human machine interface for lab abc computer applications",
             "A survey of user opinion of computer system response time",
             "The EPS user interface management system",
             "System and human system engineering testing of EPS",
             "Relation of user perceived response time to error measurement",
             "The generation of random binary unordered trees",
             "The intersection graph of paths in trees",
             "Graph minors IV Widths of trees and well quasi ordering",
             "Graph minors A survey"]

# remove common words and tokenize
stoplist = set('for a of the and to in'.split())
texts = [[word for word in document.lower().split() if word not in stoplist]
         for document in documents]

# remove words that appear only once
all_tokens = sum(texts, [])
tokens_once = set(word for word in set(all_tokens) if all_tokens.count(word) == 1)

texts = [[word for word in text if word not in tokens_once] for text in texts]

dictionary = corpora.Dictionary(texts)
corp = [dictionary.doc2bow(text) for text in texts]

tfidf = models.TfidfModel(corp) # step 1 -- initialize a model
corpus_tfidf = tfidf[corp]

# extract 400 LSI topics; use the default one-pass algorithm
lsi = models.lsimodel.LsiModel(corpus=corp, id2word=dictionary, num_topics=2)

corpus_lsi = lsi[corpus_tfidf]


#for i in range(0, lsi.num_topics-1):
for i in range(0, 3):
    print lsi.print_topics(i)

for doc in corpus_lsi: # both bow->tfidf and tfidf->lsi transformations are actually executed here, on the fly
    print(doc)
4

1 回答 1

3

corpus_lsi 有一个包含 9 个向量的列表，即文档的数量。每个向量在其第 i 个索引处存储该文档属于主题 i 的可能性。如果您只想将文档分配给 1 个主题，请选择向量中具有最高值的主题索引。

于 2016-11-24T12:48:40.483 回答