1

我想通过 SparkR shell 在 yarn-client 上运行 sparkR。所以我这样做:

./sparkR

sparkR.stop();
sc <- sparkR.init(master="yarn-client",appName="SparkR-Parquet-example2", sparkHome = Sys.getenv("SPARK_HOME"),sparkExecutorEnv = list(HADOOP_CONF_DIR=”/etc/hadoop/conf.cloudera.yarn”,YARN_CONF_DIR=”/etc/hadoop/conf.cloudera.yarn”))
sqlContext <- sparkRSQL.init(sc)
path<-"hdfs://year=2015/month=1/day=9"
AppDF <- parquetFile(sqlContext, path)

Error in callJMethod(sqlContext, "parquetFile", paths) : 
Invalid jobj 1. If SparkR was restarted, Spark operations need to be re-executed.

我是新手,有人可以帮忙解决吗?

我正在使用 spark-1.4.0-bin-hadoop2.6

4

0 回答 0