3

我想将 kudu 与 pyspark 一起使用。虽然我可以使用它:

sc.read.format('org.apache.kudu.spark.kudu').option('kudu.master',"hdp1:7051").option('kudu.table',"impala::test.z_kudu_tab").load() 

我找不到导入 KuduContext 的方法。我正在使用 jupyter 笔记本,并将其导入:

os.environ["PYSPARK_SUBMIT_ARGS"] = "--driver-memory 2g --packages com.ibm.spss.hive.serde2.xml:hivexmlserde:1.0.5.3 --packages org.apache.kudu:kudu-spark2_2.11:1.7.0 pyspark-shell"

我不工作的代码:

kudu_Context = KuduContext("es2-hdp1:7051", sc)

死于错误:

NameError: name 'KuduContext' is not defined

我也试过:

kudu_context = sc._jvm.org.apache.kudu.spark.kudu.KuduContext("hdp1:7051", sc.sparkContext)

死于错误:

AttributeError: 'SparkContext' object has no attribute '_get_object_id'
4

0 回答 0