我有一个 SQL,我正在尝试使用数据帧在 spark scala 中编码
SELECT country,
Substr(substring_col, 1, 3) AS Code,
CASE
WHEN Substr(substring_col, 1, 9) = '238208700' THEN
'columnName1'
WHEN Substr(substring_col, 1, 9) = '240018000' THEN 'columnName2'
WHEN Substr(substring_col, 1, 9) = '240017531' THEN 'columnName3'
WHEN Substr(substring_col, 1, 9) = '240017001'
OR Substr(substring_col, 1, 9) = '240017301'
OR Substr(substring_col, 1, 9) = '240017302' THEN 'columnName4'
WHEN Substr(substring_col, 1, 9) = '240017211' THEN 'columnName5'
WHEN Substr(substring_col, 1, 9) = '248010160'
OR Substr(substring_col, 1, 9) = '248010241'
OR Substr(substring_col, 1, 9) = '248010420' THEN 'columnName6'
ELSE 'not_defined'
END AS custom_column_name
FROM t_filtered
下面是我试图让它看起来很漂亮但无法达到预期结果的代码
val substring_col = substring(col("col_name"),1,9)
val mapper = Map("columnName1" -> List("238208700"),
"columnName2" -> List("240017301", "240017001")
)
myDf.withColumn("custom_column_name" ,mapper.foldLeft(lit(""))((accu, mapperMap) => {
when(substring_col isin mapperMap._2, mapperMap._1)
}))
我收到以下错误
Unsupported literal type class scala.collection.immutable.$colon$colon List(238208700)
java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(238208700)