我有来自 Kafka 的三个传入流。我解析作为 JSON 接收的流并将它们提取到适当的案例类并形成以下模式的 DStream:
case class Class1(incident_id: String,
crt_object_id: String,
source: String,
order_number: String)
case class Class2(crt_object_id: String,
hangup_cause: String)
case class Class3(crt_object_id: String,
text: String)
我想加入这三个基于公共列的 DStream 即crt_object_id
. 所需的 DStream 应采用以下形式:
case class Merged(incident_id: String,
crt_object_id: String,
source: String,
order_number: String,
hangup_cause: String,
text: String)
请告诉我一种方法来做同样的事情。我对 Spark 和 Scala 都很陌生。