1

我正在尝试使用大查询数据传输 api 安排查询并提供所需的权限 bigquery.admin 并启用大查询传输 api。权限文档： https ://cloud.google.com/bigquery-transfer/docs/enable-transfer-service 还尝试了项目所有者对服务帐户的权限。但仍然给出同样的错误。

代码文档：（使用服务帐户设置计划查询） https://cloud.google.com/bigquery/docs/scheduling-queries

出现错误的部分

 transfer_config = transfer_client.create_transfer_config(
            bigquery_datatransfer.CreateTransferConfigRequest(
                parent=parent,
                transfer_config=transfer_config,
                service_account_name=service_account_name,
            )
        )

错误堆栈跟踪

Traceback (most recent call last):
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 73, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.PERMISSION_DENIED
        details = "The caller does not have permission"
        debug_error_string = "{"created":"@1633536014.842657676","description":"Error received from peer ipv4:142.250.192.138:443","file":"src/core/lib/surface/call.cc","file_line":1070,"grpc_message":"The caller does not have permission","grpc_status":7}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "__main__.py", line 728, in <module>
    mbc.schedule_query()
  File "/home/ubuntu/prod/trell-ds-framework/data_engineering/data_migration/schedule_quries.py", line 62, in schedule_query
    service_account_name=service_account_name,
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/google/cloud/bigquery_datatransfer_v1/services/data_transfer_service/client.py", line 647, in create_transfer_config
    response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py", line 145, in __call__
    return wrapped_func(*args, **kwargs)
  File "/home/ubuntu/prod/venv_trellai/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 75, in error_remapped_callable
    six.raise_from(exceptions.from_grpc_error(exc), exc)
  File "<string>", line 3, in raise_from
google.api_core.exceptions.PermissionDenied: 403 The caller does not have permission

服务文件具有以下所有这些凭据。

BigQuery 管理员

BigQuery 数据传输服务代理

服务帐户令牌创建者

存储管理员

我已经在环境变量中设置了 json 身份验证凭据，但仍然给出权限错误。

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = Constants.BIG_QUERY_SERVICE_ACCOUNT_CRED

有谁可以帮我离开这里吗？提前致谢。

4

2 回答 2

1

看看这个关于身份验证的页面：https ://cloud.google.com/bigquery/docs/authentication/service-account-file#python

假设您使用的是服务帐户，您可以明确提供凭据以确认它们按预期工作：

from google.cloud import bigquery
from google.oauth2 import service_account

# TODO(developer): Set key_path to the path to the service account key
#                  file.
# key_path = "path/to/service_account.json"

credentials = service_account.Credentials.from_service_account_file(
    key_path, scopes=["https://www.googleapis.com/auth/cloud-platform"],
)

client = bigquery.Client(credentials=credentials, project=credentials.project_id,)
于 2021-10-06T18:17:23.533 回答
0

我建议您查看您使用的服务帐户是否指的是您正在使用的项目，并且是否具有安排查询所需的所有权限。我最好的猜测是您使用服务帐户指向另一个项目。

此外，您需要为服务帐户添加一个额外角色，即下一个“服务帐户令牌创建者”。

于 2021-10-07T17:37:23.950 回答