我正在一个大查询表上设置谷歌 DLP 扫描,以查找可识别的个人信息。我一直在研究谷歌示例代码,但是代码的 pub/sub 元素有问题
这是一个调用 google dlp 的 python 谷歌云函数,这里使用谷歌示例,使用方法 inspect_bigquery。
...
actions = [{
'pub_sub': {'topic': '{}/topics/{}'.format(parent, topic_id)},
'save_findings': {
'output_config': {
'table': {
'project_id': project,
'dataset_id': dataset_id,
'table_id': table_id + '_inspection_results',
}
}
},
}]
...
subscriber = google.cloud.pubsub.SubscriberClient()
subscription_path = subscriber.subscription_path(
project, subscription_id)
# subscription = subscriber.subscribe(subscription_path, callback)
subscription = subscriber.subscribe(subscription_path)
...
def callback(message):
try:
if (message.attributes['DlpJobName'] == operation.name):
# This is the message we're looking for, so acknowledge it.
message.ack()
# Now that the job is done, fetch the results and print them.
job = dlp.get_dlp_job(operation.name)
if job.inspect_details.result.info_type_stats:
for finding in job.inspect_details.result.info_type_stats:
print('Info type: {}; Count: {}'.format(
finding.info_type.name, finding.count))
else:
print('No findings.')
# Signal to the main thread that we can exit.
job_done.set()
else:
# This is not the message we're looking for.
message.drop()
except Exception as e:
# Because this is executing in a thread, an exception won't be
# noted unless we print it manually.
print(e)
raise
# Register the callback and wait on the event.
subscription.open(callback)
finished = job_done.wait(timeout=timeout)
if not finished:
print('No event received before the timeout. Please verify that the '
'subscription provided is subscribed to the topic provided.')
我遇到了两个错误,当我只使用订阅路径离开订阅方法时,它会出现 TypeError 错误:订阅()缺少 1 个必需的位置参数:“回调”。
当我将回调放入 subscribe 方法时,它失败,函数执行耗时 60002 毫秒,状态为:'timeout' 在超时之前未收到任何事件。请验证提供的订阅是否订阅了提供的主题。
但是,保存结果操作确实有效,几秒钟后我可以在 bigquery 中看到结果。
谢谢