2

我正在尝试使用 spark-csv_2.10-1.3.0 将 csv 文件加载到数据帧

sqlContext.read.format('com.databricks.spark.csv')
               .options(header='true',dateFormat='dd/MM/YYYY hh:mm')
               .load('test.csv',schema = s_acr_rv_derivative_schema)

架构:

s_acr_rv_derivative_schema = StructType([
        StructField("COUNTERPARTY", StringType(), True),
        StructField("ORIG_COUNTERPARTY", StringType(), True),
        StructField("CUSTOMER_NAME", StringType(), True),
        StructField("TRADE_ID", StringType(), True),
        StructField("TRADE_DATE", DateType(), True)]

问题:

Caused by: java.lang.IllegalArgumentException
        at java.sql.Date.valueOf(Date.java:140)
        at com.databricks.spark.csv.util.TypeCast$.castTo(TypeCast.scala:64)
4

0 回答 0