使用我的函数的以下代码计算查询与数据的余弦相似度:
def rank_retrieve(self, query):
"""
Given a query (a list of words), return a rank-ordered list of
documents and score for the query.
self.docs : list of documents
self.docs[i] : list of words in doc number i -> [word1,word2,...,wordN]
self.boolean_retrieve(query) : giving a list of words this return the index of
documents wich contains all of these words.
self.tfidf(word,documentIndex) : returns the value tfidf of a word in a document
self.get_posting(word): returns a list of document index where that word appears
"""
scores = [0.0 for xx in range(len(self.docs))]
# Apply Cosine Similarity
for i in self.boolean_retrieve(query):
normDoc = 0.0
normQuery = 0.0
dt = 0.0
qtdt = 0.0
for word in query:
dt = self.get_tfidf(word,i)
normDoc+= math.pow(dt,2)
qt = 1.0 + ( math.log10( len(query) ) )
normQuery+=math.pow(qt,2)
qtdt += dt * qt
scores[i] = qtdt / ( math.sqrt(normDoc) )
return scores
我只有下一个理论:
那么,你能帮我写代码吗?我返回错误的值,我不知道为什么。谢谢。
doc 56 评分结果:
Cosine Similarity Test
doc 56, query : ['separ', 'of', 'church', 'and', 'state']
Separ:
QTDT: 0.105587429399
DT 0.0621479067488
QT 1.69897000434
NormDoc: 0.00386236231326 normQuery 2.88649907563
Of:
QTDT: 0.105587429399
DT 0.0
QT 1.69897000434
NormDoc: 0.00386236231326 normQuery 5.77299815127
Church :
QTDT: 0.653857934128
DT 0.322707583613
QT 1.69897000434
NormDoc: 0.108002546834 normQuery 8.6594972269
And:
QTDT: 0.653857934128
DT 0.0
QT 1.69897000434
NormDoc: 0.108002546834 normQuery 11.5459963025
State:
QTDT: 0.674927180008
DT 0.0124011876763
QT 1.69897000434
NormDoc: 0.10815633629 normQuery 14.4324953782
Scores of 56 must be 0.010676611271744128 found : 2.05225316563