2

GridSearchCV 的文档指出我可以通过评分功能。

评分：字符串，可调用或无，默认=无

我想使用原生的accuracy_score作为评分函数。

所以这是我的尝试。进口和一些数据：

import numpy as np
from sklearn.cross_validation import KFold, cross_val_score
from sklearn.grid_search import GridSearchCV
from sklearn.metrics import accuracy_score
from sklearn import neighbors

X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
Y = np.array([0, 1, 0, 0, 0, 1])

现在，当我只使用没有评分功能的 k 折交叉验证时，一切都按预期工作：

parameters = {
    'n_neighbors': [2, 3, 4],
    'weights':['uniform', 'distance'],
    'p': [1, 2, 3]
}
model = neighbors.KNeighborsClassifier()
k_fold = KFold(len(Y), n_folds=6, shuffle=True, random_state=0)
clf = GridSearchCV(model, parameters, cv=k_fold)  # TODO will change
clf.fit(X, Y)

print clf.best_score_

但是当我将线路更改为

clf = GridSearchCV(model, parameters, cv=k_fold, scoring=accuracy_score) # or accuracy_score()

我得到了错误：ValueError: Cannot have number of folds n_folds=10 greater than the number of samples: 6.我认为这并不代表真正的问题。

在我看来，问题在于accuracy_score不遵循scorer(estimator, X, y)文档中所写的签名

那么我该如何解决这个问题呢？

4

2 回答 2

5

如果您更改scoring=accuracy_score为scoring='accuracy'（请参阅文档以获取您可以通过这种方式按名称使用的完整记分员列表。）

理论上，您应该能够像您尝试的那样传递自定义评分函数，但我猜您是对的并且accuracy_score没有正确的 API。

于 2016-08-04T05:45:33.357 回答
0

这是一个使用加权 Kappa 作为简单随机森林模型的 GridSearchCV 评分指标的示例。对我来说，关键的学习是在“make_scorer”函数中使用与记分器相关的参数。

from sklearn.model_selection import GridSearchCV
from sklearn.metrics import cohen_kappa_score, make_scorer


kappa_scorer = make_scorer(cohen_kappa_score,weights="quadratic")
# Create the parameter grid based on the results of random search 
param_grid = {
    'bootstrap': [True],
    'max_features':  range(2,10), # try features from 2 to 10
    'min_samples_leaf': [3, 4, 5],
    'n_estimators' : [100,300,500],
    'max_depth':  [5]
    }
# Create a based model
random_forest = RandomForestClassifier(class_weight ="balanced_subsample",random_state=1)
# Instantiate the grid search model
grid_search = GridSearchCV(estimator = random_forest, param_grid = param_grid, 
                         cv = 5, n_jobs = -1, verbose = 2, scoring = kappa_scorer) # search for best model using roc_auc

# Fit the grid search to the data
grid_search.fit(final_tr, yTrain)
于 2020-04-30T12:55:55.757 回答