我有这个:
import spark.implicits._
import org.apache.spark.sql.catalyst.encoders.RowEncoder
val mydata: Dataset[Row] = spark.read.format("csv").option("header", true).option("inferSchema", true).load("mydata.csv")
// CSV header: Time,Area,City
// CSV values: "2016-01","A1","NY"
// "2016-01","AB","HK" etc
// ...somewhere in my aggregate:
def bufferEncoder: Encoder[Array[(String, Row)]] = ....
对于Array
我可以写的内部元组:
val rowEncoder = RowEncoder(mydata.schema)
Encoders.tuple(Encoders.STRING, rowEncoder)
但是我怎样才能Encoder
为外部写Array
呢?