2

我在使用 fore each 并行运行 randomForest 时遇到问题。看这个例子，我创建了一些数据，然后是一个公式符号。该公式本身适用于随机森林。但是在 foreach 并行循环中使用时失败......？

# rf on big training set
# use parallel foreach
library(foreach)

library(doMC)
registerDoMC(4)  #change the 2 to your number of CPU cores 
# info on parrallell backend
getDoParName()
getDoParWorkers()

# bogus data
set.seed(123)
ssize <- 100000
x1 <- sample( LETTERS[1:9], ssize, replace=TRUE, prob=c(0.1, 0.2, 0.15, 0.05,0.1, 0.2, 0.05, 0.05,0.1) )
x2 <- rlnorm(ssize,0,0.25)
x3 <- rlnorm(ssize,0,0.5)
y <- sample( c("Y","N"), ssize, replace=TRUE, prob=c(0.05, 0.95))
df <- data.frame(x1,x2,x3,y)
df$p_y <- as.numeric(df$y)-1

# use strata to sample whole dataset
library(sampling)

s1 = strata(df,stratanames = "y", size = c(2500,2500))
s2 = strata(df,stratanames = "y", size = c(2500,2500))
s3 = strata(df,stratanames = "y", size = c(2500,2500))
s4 = strata(df,stratanames = "y", size = c(2500,2500))

s_list <- list(s1$ID_unit, s2$ID_unit, s3$ID_unit, s4$ID_unit)

# model function
rf.formula <- as.formula(paste("y","~",paste("x1","x2",sep="+")))

library(randomForest)

# simple stuff works but takes some time
model.rf <-randomForest(y ~ x1 + x2, df, ntree=100, nodesize = 50)

# build rf with dopar on explicit formula works and is quick
model.rf.dopar <- foreach(subset=s_list, .combine=combine, .packages='randomForest') %dopar%
  randomForest(y ~ x1 + x2, df, ntree=100, nodesize = 50, subset=subset)

# build rf with dopar on rf.formula fails
model.rf.s.b2 <- foreach(subset=s_list, .combine=combine, .packages='randomForest') %dopar%
  randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=subset)

# > model.rf.s.b2 <- foreach(subset=s_list, .combine=combine, .packages='randomForest') %dopar%
#   +   randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=subset)
# Error in randomForest(rf.formula, df, ntree = 100, nodesize = 50, subset = subset) : 
#   task 1 failed - "invalid subscript type 'closure'"

错误：

model.rf.s.b2 <- foreach(subset=s_list, .combine=combine, .packages='randomForest') %dopar%
   +   randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=subset)

Error in randomForest(rf.formula, df, ntree = 100, nodesize = 50, subset = subset) : 
task 1 failed - "invalid subscript type 'closure'"

有什么建议么？

发送

4

1 回答 1

2

问题似乎是由于model.frame.default函数深处的索引操作出错，由randomForest.formula. 我完全不确定是什么触发了这个问题，因为在 中发生了很多棘手的评估model.frame.default，但修改公式的环境似乎可以解决问题：

r <- foreach(subset=s_list, .combine='combine', .multicombine=TRUE,
             .packages='randomForest') %dopar% {
  environment(rf.formula) <- environment()
  randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=subset)
}

特别是，这会导致subset正确评估，否则它会评估subset函数。我尝试重命名迭代变量，但没有帮助。

请注意，我也设置.multicombine为TRUErandomForestcombine函数接受多个对象，这可以显着提高性能。

更新

该问题可以通过以下方式重现：

fun <- function(subset) {
  randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=subset)
}
fun(s_list[[1]])

例如，如果将变量subset更改为s，它也会失败，但会出现误导性较小的错误消息：

> fun <- function(s) {
>   randomForest(rf.formula, df, ntree=100, nodesize = 50, subset=s)
> }
> fun(s_list[[1]])
Error in eval(expr, envir, enclos) : object 's' not found
Calls: fun ... eval -> model.frame -> model.frame.default -> eval -> eval
Execution halted

与foreach示例一样，重置公式的环境似乎可以解决问题。

于 2013-09-22T05:01:44.010 回答