我想将 kudu 与 pyspark 一起使用。虽然我可以使用它:
sc.read.format('org.apache.kudu.spark.kudu').option('kudu.master',"hdp1:7051").option('kudu.table',"impala::test.z_kudu_tab").load()
我找不到导入 KuduContext 的方法。我正在使用 jupyter 笔记本,并将其导入:
os.environ["PYSPARK_SUBMIT_ARGS"] = "--driver-memory 2g --packages com.ibm.spss.hive.serde2.xml:hivexmlserde:1.0.5.3 --packages org.apache.kudu:kudu-spark2_2.11:1.7.0 pyspark-shell"
我不工作的代码:
kudu_Context = KuduContext("es2-hdp1:7051", sc)
死于错误:
NameError: name 'KuduContext' is not defined
我也试过:
kudu_context = sc._jvm.org.apache.kudu.spark.kudu.KuduContext("hdp1:7051", sc.sparkContext)
死于错误:
AttributeError: 'SparkContext' object has no attribute '_get_object_id'