我正在使用 Dask 导入一个非常大的 csv 文件 ~680GB,但是,输出不是我所期望的。我的目标是只选择一些列(6/50),并可能过滤它们(我不确定,因为似乎没有数据?):
import dask.dataframe as dd
file_path = "/Volumes/Seagate/Work/Tickets/Third ticket/Extinction/species_all.csv"
cols = ['year', 'species', 'occurrenceStatus', 'individualCount', 'decimalLongitude', 'decimalLatitde']
dataset = dd.read_csv(file_path, names=cols,usecols=[9, 18, 19, 21, 22, 32])
当我将它读入 Jupyter 时,我无法理解输出 - 控制台输出:
Dask DataFrame Structure:
year species occurrenceStatus individualCount decimalLongitude decimalLatitde
npartitions=11397
object object object object object object
... ... ... ... ... ...
... ... ... ... ... ... ...
... ... ... ... ... ...
... ... ... ... ... ...
Dask Name: read-csv, 11397 tasks