1

我有一个管道:

np.random.seed(42)
tf.random.set_seed(42)

pipeline = Pipeline([
    ('smote', SMOTE()),
    ('under',RandomUnderSampler()),
    ('cl', KerasClassifier(build_fn=create_model, verbose=0))
])

param_grid_pipeline = {
    'smote__sampling_strategy':[.3],
    'smote__random_state':[42],
    'under__sampling_strategy':['auto'],
    'under__random_state':[42],   
    'cl__batch_size':[128],
    'cl__epochs':[20],
}

cv = StratifiedShuffleSplit(n_splits=40, test_size=0.2, random_state=42)
grid = GridSearchCV(estimator=pipeline, param_grid=param_grid_pipeline, cv=cv, scoring='f1', verbose=1, n_jobs=-1)
grid_result = grid.fit(X,y)
print("best_score_",grid_result.best_score_)

而 best_score_ 是 0.9981313067607172

但是,如果我从管道中排除重新采样并在外部执行它:

np.random.seed(42)
tf.random.set_seed(42)

over = SMOTE(sampling_strategy=0.3,random_state=42)
under = RandomUnderSampler(sampling_strategy='auto',random_state=42)
X,y = over.fit_resample(X,y)
X,y = under.fit_resample(X,y)

pipeline = Pipeline([
    ('cl', KerasClassifier(build_fn=create_model, verbose=0))
])

param_grid_pipeline = {
    'cl__batch_size':[128],
    'cl__epochs':[20],
}

cv = StratifiedShuffleSplit(n_splits=40, test_size=0.2, random_state=42)
grid = GridSearchCV(estimator=pipeline, param_grid=param_grid_pipeline, cv=cv, scoring='f1', verbose=1, n_jobs=-1)
grid_result = grid.fit(X,y)
print("best_score_",grid_result.best_score_)

而且我(在多次运行中)得到了更好的结果:0.9999888503305302

在管道外部使用重新采样有什么区别?

4

1 回答 1

0

如果您在外部执行过采样和欠采样,您将在整个数据集上执行它,然后才将其拆分以进行交叉验证(这解释了更高的分数,因为在测试集中有关于您的训练集的“信息”)!如果将其包含在管道中,则只需对训练集重新采样(这实际上是正确的方法)。尝试手动进行交叉验证的随机拆分,然后仅在训练集上执行重采样,然后手动计算分数(因为在此示例中您实际上没有参数网格,这不是很多工作)和那么你应该得到相同的结果。

于 2020-05-07T20:18:48.860 回答