0

What is the ...?我使用/fcselect没有排名 ID 的处理程序问了一个问题，并获得了以下文档：

"docs": [
  {"id": "100"},  // ranked first
  {"id": "101"},  // ranked second
  ...
  {"id": "198"},  // ranked second from last (99th)
  {"id": "199"}   // ranked last (100th)
]

然后，我使用以下基本事实创建了一个排名器：

What is the ...?,199,5,198,4
...

然后，我使用带有排名器的处理程序问了*相同的问题* ，并获得了以下文档：/fcselect

"docs": [
  {"id": "100"},  // ranked first
  {"id": "101"},  // ranked second
  ...
  {"id": "199"},  // ranked 30th
  ...
  {"id": "198"}   // ranked 35th
  ...
]

但我希望像以下顺序：

"docs": [
  {"id": "199"},  // ranked first
  {"id": "198"},  // ranked second
  {"id": "100"},  // ranked third
  {"id": "101"}   // ranked 4th
  ...
]

排名者是否经过适当培训？

4

1 回答 1

3

See here for an answer to a similar question: https://developer.ibm.com/answers/questions/317822/4-stars-answers.html.

With a learning-to-rank approach there are certainly no guarantees that the ranker will move answers marked as 'correct' in the ground truth to the top of the search result. This is because the ranker is not memorizing correct answers, but rather hoping to capture generalizations in the feature value distributions capturing the overlap between queries and search results. To validate ranker training is behaving as expected, you can measure top 1 result accuracy over a large set of queries (different from the queries used during training) and check for an improvement on average.

That said, it is certainly strange that the top two ranked search results do not change at all in response to ranker training. Some things to consider in experimentation to improve performance:

  • Are the number of rows being passed to the ranker sufficiently high (in your example, the parameter should be set to at least 100 since the default is 10).
  • Are the number of rows included during ranker training (when preparing the ground truth file) the same as the number of rows included during runtime (they should match for optimal performance - and it's a setting you can play with for tuning performance)?
  • Is there a lexical gap between the query and the correct answer documents that's likely to confuse the ranker? Could synonyms / stopword removal / lowercasing / stemming etc be incorporated into your index/query analyzers to improve overlap between the query and correct answer?
  • Are there additional features you could add and pass to the ranker during training and runtime that might be better able to capture the overlap between questions and candidate answers from the search result? See here for more info: https://medium.com/machine-learning-with-ibm-watson/developing-with-ibm-watson-retrieve-and-rank-part-3-custom-features-826fe88a5c63?cm_mc_uid=06908383978514879641730&cm_mc_sid_50200000=1488383112#.gtzsdg4k3
于 2017-03-01T16:02:12.820 回答