我想将数据拆分为分层的训练、测试和验证数据集,但 sklearn 仅提供 cross_validation.train_test_split 只能分为 2 部分。如果我想这样做,我应该怎么做
问问题
5606 次
2 回答
5
如果您想使用 Stratified Train/Test 拆分,可以在 Sklearn 中使用 StratifiedKFold
假设X
是您的特征,并且y
是您的标签,基于此处的示例:
from sklearn.model_selection import StratifiedKFold
cv_stf = StratifiedKFold(n_splits=3)
for train_index, test_index in skf.split(X, y):
print("TRAIN:", train_index, "TEST:", test_index)
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
更新:要将数据拆分为 3 个不同的百分比,使用numpy.split()可以这样做:
X_train, X_test, X_validate = np.split(X, [int(.7*len(X)), int(.8*len(X))])
y_train, y_test, y_validate = np.split(y, [int(.7*len(y)), int(.8*len(y))])
于 2017-09-15T05:57:50.800 回答
1
您也可以train_test_split
多次使用来实现此目的。第二次,在第一次调用的训练输出上运行它train_test_split
。
from sklearn.model_selection import train_test_split
def train_test_validate_stratified_split(features, targets, test_size=0.2, validate_size=0.1):
# Get test sets
features_train, features_test, targets_train, targets_test = train_test_split(
features,
targets,
stratify=targets,
test_size=test_size
)
# Run train_test_split again to get train and validate sets
post_split_validate_size = validate_size / (1 - test_size)
features_train, features_validate, targets_train, targets_validate = train_test_split(
features_train,
targets_train,
stratify=targets_train,
test_size=post_split_validate_size
)
return features_train, features_test, features_validate, targets_train, targets_test, targets_validate
于 2018-09-28T16:10:44.763 回答