3

我对 scikit-learn 很陌生，我正在尝试使用这个包来预测收入数据。这可能是一个重复的问题，因为我看到了另一篇关于此的帖子，但我正在寻找一个简单的示例来了解 scikit-learn 估计器的预期。

我拥有的数据具有以下结构，其中许多特征是分类的（例如：工作班、教育......）

age: continuous.
workclass: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.
fnlwgt: continuous.
education: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.
education-num: continuous.
marital-status: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.
occupation: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.
relationship: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.
race: White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black.
sex: Female, Male.
capital-gain: continuous.
capital-loss: continuous.
hours-per-week: continuous.
native-country: United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.

示例记录：

38   Private    215646   HS-grad    9    Divorced    Handlers-cleaners   Not-in-family   White   Male   0   0   40   United-States   <=50K
53   Private    234721   11th   7    Married-civ-spouse  Handlers-cleaners   Husband     Black   Male   0   0   40   United-States   <=50K
30   State-gov  141297   Bachelors  13   Married-civ-spouse  Prof-specialty  Husband     Asian-Pac-Islander  Male   0   0   40   India   >50K

我很难处理分类特征，因为 sckit-learn 中的大多数模型都希望所有特征都是数字？他们确实提供了一些类来转换/编码这些功能（如 Onehotencoder、DictVectorizer），但我找不到在我的数据上使用这些的方法。我知道在我将它们完全编码为数字之前，这里涉及到很多步骤，但我只是想知道是否有人知道一种更简单有效（因为有太多这样的功能）的方式，可以通过一个例子来理解。我隐约知道 DictVectorizer 是要走的路，但需要帮助才能在这里进行。

4

1 回答 1

6

这是一些使用DictVectorizer. 首先，让我们在 Python shell 中设置一些数据。我将文件的阅读留给你。

>>> features = ["age", "workclass", "fnlwgt", "education", "education-num", "marital-status", "occupation",
...             "relationship", "race", "sex", "capital-gain", "capital-loss", "hours-per-week", "native-country"]
>>> input_text = """38   Private    215646   HS-grad    9    Divorced    Handlers-cleaners   Not-in-family   White   Male   0   0   40   United-States   <=50K
... 53   Private    234721   11th   7    Married-civ-spouse  Handlers-cleaners   Husband     Black   Male   0   0   40   United-States   <=50K
... 30   State-gov  141297   Bachelors  13   Married-civ-spouse  Prof-specialty  Husband     Asian-Pac-Islander  Male   0   0   40   India   >50K
... """

现在，解析这些：

>>> for ln in input_text.splitlines():
...     values = ln.split()
...     y.append(values[-1])
...     d = dict(zip(features, values[:-1]))
...     samples.append(d)

我们现在得到了什么？让我们检查：

>>> from pprint import pprint
>>> pprint(samples[0])
{'age': '38',
 'capital-gain': '0',
 'capital-loss': '0',
 'education': 'HS-grad',
 'education-num': '9',
 'fnlwgt': '215646',
 'hours-per-week': '40',
 'marital-status': 'Divorced',
 'native-country': 'United-States',
 'occupation': 'Handlers-cleaners',
 'race': 'White',
 'relationship': 'Not-in-family',
 'sex': 'Male',
 'workclass': 'Private'}
>>> print(y)
['<=50K', '<=50K', '>50K']

这些samples都准备好了DictVectorizer，所以传递它们：

>>> from sklearn.feature_extraction import DictVectorizer
>>> dv = DictVectorizer()
>>> X = dv.fit_transform(samples)
>>> X
<3x29 sparse matrix of type '<type 'numpy.float64'>'
        with 42 stored elements in Compressed Sparse Row format>

等等，你有X并且y可以传递给估计器，只要它支持稀疏矩阵。（否则，传递sparse=False给DictVectorizer构造函数。）

测试样本可以类似地传递给DictVectorizer.transform; 如果测试集中没有出现在训练集中的特征/值组合，这些将被简单地忽略（因为学习模型无论如何都无法对它们做任何明智的事情）。

于 2013-10-08T09:10:31.703 回答