rapidminer - problems of loading large csv file into rapidminer

Question

I am trying to load a big csv file(about 18G) into rapidminer for building a classification model. The “import configuration wizard” seems has difficulty in loading the data. Therefore, I choose to use the “Edit parameter list: data set meta data information” to set up the attribute and label information. However, the UI-interface only allows me to setup those information column-by-column. My csv file has about 80000 columns. How should I handle this kind of scenario? Thanks.

Answer 1

我自己还没有尝试过，但是您应该能够将 CSV 加载到 MySQL 数据库中。然后，您可以使用流数据库运算符来避免大小限制。以下是 RapidMiner 的描述：

与将数据加载到主存中的Read Database算子相比，Stream Database算子将数据保存在数据库中并进行批量读取数据。这允许 RapidMiner 访问任意大小的数据集而没有任何大小限制。