python - 如何在pydeequ中传递直方图分箱udf？

Question

我无法将 binningudf 传递给 pydeequ 的直方图分析器。有人可以帮忙吗

import pyspark
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, udf
from pyspark.sql.types import StringType, IntegerType

spark = SparkSession.builder.appName('SparkByExamples.com').getOrCreate()

columns = ["Seqno","Name"]
data = [("1", "john jones"),
    ("3", "tracey smith"),
    ("3", "amy sanders"),("2", "amy sanders"),("4", "amy sanders"),("5", "amy sanders")]

df = spark.createDataFrame(data=data,schema=columns)

def createBucket(x):
    if int(x)>2: return 1
    else: return 2

createBucketUdf = udf(lambda z: createBucket(z), IntegerType())

analysisResult = AnalysisRunner(spark).onData(df) \
                    .addAnalyzer(Histogram("SeqNo",binningUdf=lambda z: createBucket(z))) \
                    .run()
                    
result_df = AnalyzerContext.successMetricsAsDataFrame(spark, analysisResult)
result_df.show()

上面的代码抛出错误说

AttributeError: 'function' object has no attribute '_get_object_id'

似乎上述错误是由于参数类型不正确。所以我尝试将 lambda 作为字符串。但后来我得到以下回应

+------+--------+----+-----+
|entity|instance|name|value|
+------+--------+----+-----+
+------+--------+----+-----+