r - Error: Invalid: File is too small to be a well-formed file - error when using feather in R

Question

I'm trying to use feather (v. 0.0.1) in R to read a fairly large (3.5 GB) csv file with 21178665 rows and 16 columns.

I use the following lines to load the file:

library(feather)
path <- "pp-complete.csv"
df <- read_feather(path)

But I get the following error:

Error: Invalid: File is too small to be a well-formed file

There's no explanation in the documentation of read_feather so I'm not sure what's the problem. I guess this function expects a different file form but I'm not sure what that would be.

Btw, I can read the file with read_csv in readr library but it takes a while.

Answer 1

feather文件格式不同于 CSV 文件格式。它们不可互换。该read_feather函数无法读取简单的 CSV 文件。

如果您想快速阅读 CSV 文件，最好的选择可能是readr::read_csv或data.table::fread. 对于大文件，从光盘读取它通常仍需要一段时间。

将数据加载到 R 后，您可以创建feather格式为的文件，write_feather以便下次读取read_feather。