r - Any way to force fread() of data.table not to stop on empty lines?

Question

(question is not relevant anymore, since new version of data.table of 25-NOV-2016 - see accepted answer below)

So, I have a table with some empty lines in the middle. When I try to open it with fread, it stops, saying Stopped reading at empty line 10006, but text exists afterwards (discarded). Is there any way to avoid this without changing the data file?

Answer 1

2016 年 11 月 25 日发布的 data.table 版本 1.9.8blank.lines.skip具有跳过空白行的新选项。

text <- "1,a\n\n2,b\n3,c\n4,a\n\n5,b\n\n6,c"

library(data.table)
fread(text)
##    V1 V2
## 1:  2  b
## 2:  3  c
## 3:  4  a
## Warning message:
## In fread("1,a\n\n2,b\n3,c\n4,a\n\n5,b\n\n6,c") :
##   Stopped reading at empty line 6 but text exists afterwards (discarded): 5,b

fread(text, blank.lines.skip=TRUE)
##    V1 V2
## 1:  1  a
## 2:  2  b
## 3:  3  c
## 4:  4  a
## 5:  5  b
## 6:  6  c

Answer 2

You can use the Windows findstr command to get rid of empty lines.

Example file "Data.txt".

1,a

2,b
3,c
4,a


5,b

6,c

Reproduces your error.

> dt <- fread("Data.txt")
Warning message:
In fread("Data.txt") :
Stopped reading at empty line 6 of file, but text exists afterwards (discarded): 5,b

But works when using Windows findstr directly in fread.

> require(data.table)
> dt <- fread('findstr "." Data.txt')

# > dt
#    V1 V2
# 1:  1  a
# 2:  2  b
# 3:  3  c
# 4:  4  a
# 5:  5  b
# 6:  6  c

Answer 3

如果其他人遇到类似问题，我注意到如果您没有明确说明，data.table 1.10.4（我正在使用的当前 2017 版本）似乎会在某些文件中产生空行错误：

'strip.white = FALSE'

我正在查看我试图导入的约 350 个文件中的明显线路错误。一些行在原件中的两行之间断开，并且由于它们包含不同形式的信息，fread 警告某些列存在阶级强制问题。但是对于几乎每个文件，在不同的行上，我同时也遇到了“空行”错误。我在记事本++中手动检查了这些。很多次。没有空行，还有剩余的行；其中很多。尝试处理导入参数并专门禁用 strip.white 删除了空行警告。