0

我正在使用https://github.com/aws/aws-mwaa-local-runner 在本地测试 Airflow

问题：无法从 Secretmanager 获取连接详细信息

AIRFLOW_CTX_DAG_EMAIL=admin@localhost.com
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=connection_test_dag
AIRFLOW_CTX_TASK_ID=snowflake_op_sql_str
AIRFLOW_CTX_EXECUTION_DATE=2021-09-16T17:52:15.503730+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-09-16T17:52:15.503730+00:00
[2021-09-16 17:52:18,562] {{snowflake.py:119}} INFO - Executing: select current_date()
[2021-09-16 17:52:18,596] {{taskinstance.py:1482}} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1138, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1311, in _prepare_and_execute_task_with_callbacks
    result = self._execute_task(context, task_copy)
  File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1341, in _execute_task
    result = task_copy.execute(context=context)
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/providers/snowflake/operators/snowflake.py", line 121, in execute
    hook.run(self.sql, autocommit=self.autocommit, parameters=self.parameters)
  File "/usr/local/lib/python3.7/site-packages/airflow/hooks/dbapi.py", line 173, in run
    with closing(self.get_conn()) as conn:
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 215, in get_conn
    conn_config = self._get_conn_params()
  File "/usr/local/airflow/.local/lib/python3.7/site-packages/airflow/providers/snowflake/hooks/snowflake.py", line 147, in _get_conn_params
    self.snowflake_conn_id  # type: ignore[attr-defined] # pylint: disable=no-member
  File "/usr/local/lib/python3.7/site-packages/airflow/hooks/base.py", line 67, in get_connection
    conn = Connection.get_connection_from_secrets(conn_id)
  File "/usr/local/lib/python3.7/site-packages/airflow/models/connection.py", line 354, in get_connection_from_secrets
    raise AirflowNotFoundException(f"The conn_id `{conn_id}` isn't defined")
airflow.exceptions.AirflowNotFoundException: The conn_id `snowflake_conn_dev` isn't defined

我尝试如下更新docker compose，但它没有成功

在此处输入图像描述

4

1 回答 1

0

从您发布的日志来看，问题似乎是您尚未配置连接“snowflake_conn_dev”（请参阅​​最后一行），而不是有关 SecretsManager 的任何连接问题。

于 2021-10-14T14:02:55.943 回答