java - What metrics can I use to validate and test RankNet in the RankLib library in the Lemur Project?

Question

I am currently using the RankLib implementation of the RankNet algorithm (-ranker 4) with a held-out set. I am using the jar file in terminal to run this.

The documentation stipulates:

metric2t (e.g. NDCG, ERR, etc) only applies to list-wise algorithms (AdaRank, Coordinate Ascent and LambdaMART). Point-wise and pair-wise techniques (MART, RankNet, RankBoost), due to their nature, always use their internal RMSE / pair-wise loss as the optimisation criteria.

However, when I set the 'metrics2t' to ERR@10 or NDCG@10, it starts to train and validate on my chosen metric rather that 'RMSE'.

This is part of the table outputted when I run RankNet with ERR@10.

Is there something that I am missing as this seems to be a contradiction to me.

Thanks.

Answer 1

我不确定，但是，我认为即使它打印了这些指标的结果，它也没有针对它们进行优化。

该库的开发人员只是将其留在那里，至于其他方法，通常使用其中一个指标进行验证。并且没有选项可以在训练期间简单地关闭计算指标。

现在我正在训练一个 RankNet 模型，似乎训练和验证数据的 ERR@10 实际上在增加，而“错误排序对的百分比”正在减少。