0

databricks我在Scala 笔记本中运行了以下代码，但出现错误。

添加库：azure-cosmosdb-spark_2.4.0_2.11-1.3.4-uber 代码：

import org.apache.spark.rdd.RDD
import org.apache.spark.{SparkConf, SparkContext}

import spark.implicits._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.Column
import org.apache.spark.sql.types.{StructType, StructField, StringType, IntegerType,LongType,FloatType,DoubleType, TimestampType}
import org.apache.spark.sql.cassandra._

//datastax Spark connector
import com.datastax.spark.connector._
import com.datastax.spark.connector.cql.CassandraConnector
import com.datastax.driver.core.{ConsistencyLevel, DataType}
import com.datastax.spark.connector.writer.WriteConf

//Azure Cosmos DB library for multiple retry
import com.microsoft.azure.cosmosdb.cassandra

import sqlContext.implicits._
spark.conf.set("x","x")
spark.conf.set("x","x")
spark.conf.set("x","x")
spark.conf.set("x","x")

val CaseFileDFTemp = sqlContext
  .read
  .format("org.apache.spark.sql.cassandra")
  .options(Map( "table" -> "case_files", "keyspace" -> "shared"))
.load().show()

CaseFileDFTemp.show()

错误：

错误：值 show 不是 Unit CaseFileDFTemp.show() 的成员

4

2 回答 2

1

Can you please try creating the SQL context and try the show function.

import sqlContext.implicits._
val sqlContext= new org.apache.spark.sql.SQLContext(sc)

Please let me know if it helps.

于 2020-01-10T07:32:15.107 回答
0

如果你写

val CaseFileDFTemp = sqlContext
  .read
  .format("org.apache.spark.sql.cassandra")
  .options(Map( "table" -> "case_files", "keyspace" -> "shared"))
.load().show()

然后CaseFileDFTemp将有 type Unit，因为show()它将“消耗”您的数据框。所以删除show()，然后它会工作

于 2020-01-10T13:38:46.327 回答