1

我有以下 xml 文件：

<?xml version="1.0" encoding="UTF-8"?>
<data>
<text>We are a doing nothing here you can say it time pass. what are you doing doing doing doing doing time?</text>
<text>We are a doing nothing here you can say it time pass. what are you doing doing doing doing doing time?</text>
</data>

现在我执行了以下查询：

let $hits :=
let $terms :=
let $node := xdmp:document-filter(doc("/content/C/Documents and Settings/vimleshm/Desktop/abc.xml"))
return 
(cts:distinctive-terms($node,
<options xmlns="cts:distinctive-terms"
xmlns:db="http://marklogic.com/xdmp/database">
<use-db-config>false</use-db-config>
<score>logtf</score>
<max-terms>100</max-terms>
<db:word-searches>true</db:word-searches>
<db:stemmed-searches>off</db:stemmed-searches>
<db:fast-phrase-searches>false</db:fast-phrase-searches>
<db:fast-element-word-searches>false</db:fast-element-word-searches>
<db:fast-element-phrase-searches>false</db:fast-element-phrase-searches>
</options>)//cts:term)
for $wq in $terms
where $wq/cts:word-query
return element word {
attribute score {                               $wq/@score},
$wq/cts:word-query/cts:text/string() }
return 

let $x:=
for $hit in $hits

return $hit
return $x

它给了我以下回应：

<?xml version="1.0" encoding="UTF-8"?>
<results warning="more than one root item">
  <word score="36864">doing</word>
  <word score="26624">text</word>
  <word score="26624">you</word>
  <word score="26624">time</word>
  <word score="26624">are</word>
  <word score="22528">a</word>
  <word score="22528">we</word>
  <word score="22528">it</word>
  <word score="22528">data</word>
  <word score="22528">can</word>
  <word score="22528">pass</word>
  <word score="22528">here</word>
  <word score="22528">nothing</word>
  <word score="22528">what</word>
  <word score="22528">say</word>
</results>

有人能告诉我这个分数 [log(term frequency)] 是如何计算的吗？上述案例中的示例在总共 42 个单词中使用了 12 次“doing”。

以下是上述文件的总条款和频率[括号内]

doing [12]
you [4]
time [4]
are [4]
a [2]
We [2]
nothing [2]
here [2]
can  [2]
say [2]
it [2]
pass [2]
what [2]
4

1 回答 1

5

http://docs.marklogic.com/guide/search-dev/relevance#chapter无疑是最好的起点。这里还有比 logTF 更多的事情。还有：

  • IDF - 这些词在数据库中有多常见？
  • 文档长度标准化 - 较长的文档往往比较短的文档有更多的词，因此分数会根据文档长度进行缩放
  • 而 logTF 实际上是阶梯式 TF 函数的自然对数（为了速度）

所有这些因素共同作用，使得分准确而快速。

于 2012-10-18T20:56:33.643 回答