0

我有一个 SQL,我正在尝试使用数据帧在 spark scala 中编码

SELECT country, 
       Substr(substring_col, 1, 3) AS Code, 
       CASE 
         WHEN Substr(substring_col, 1, 9) = '238208700' THEN 
         'columnName1' 
         WHEN Substr(substring_col, 1, 9) = '240018000' THEN 'columnName2' 
         WHEN Substr(substring_col, 1, 9) = '240017531' THEN 'columnName3' 
         WHEN Substr(substring_col, 1, 9) = '240017001' 
               OR Substr(substring_col, 1, 9) = '240017301' 
               OR Substr(substring_col, 1, 9) = '240017302' THEN 'columnName4' 
         WHEN Substr(substring_col, 1, 9) = '240017211' THEN 'columnName5' 
         WHEN Substr(substring_col, 1, 9) = '248010160' 
               OR Substr(substring_col, 1, 9) = '248010241' 
               OR Substr(substring_col, 1, 9) = '248010420' THEN 'columnName6' 
         ELSE 'not_defined' 
       END                AS custom_column_name 
FROM   t_filtered 

下面是我试图让它看起来很漂亮但无法达到预期结果的代码

val substring_col = substring(col("col_name"),1,9)
val mapper = Map("columnName1"  -> List("238208700"),
  "columnName2" -> List("240017301", "240017001")
)

myDf.withColumn("custom_column_name" ,mapper.foldLeft(lit(""))((accu, mapperMap) => {
  when(substring_col isin mapperMap._2, mapperMap._1)
}))

我收到以下错误

Unsupported literal type class scala.collection.immutable.$colon$colon List(238208700)
java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(238208700)
4

0 回答 0