2

我在 AWS ECS 中运行 Apache Airflow，在 EC2 上运行 Apache Atlas。我已经能够将 Apache Airflow 的本地实例连接到 EC2 上的 Apache Atlas；但是，我无法连接我的 AWS ECS 实例和 EC2 实例。当 DAG 中的 Airflow 任务尝试将信息推送到 Apache Atlas 时，我收到以下错误。

[2021-02-18 18:49:37,301] {connectionpool.py:752} WARNING - Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2e87410>, 'Connection to <ip-address> timed out. (connect timeout=10)')': /api/atlas/v2/types/typedefs
[2021-02-18 18:49:47,302] {connectionpool.py:752} WARNING - Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2e87b10>, 'Connection to <ip-address> timed out. (connect timeout=10)')': /api/atlas/v2/types/typedefs
[2021-02-18 18:49:57,311] {connectionpool.py:752} WARNING - Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2e9f190>, 'Connection to <ip-address> timed out. (connect timeout=10)')': /api/atlas/v2/types/typedefs
[2021-02-18 18:50:07,319] {connectionpool.py:752} WARNING - Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2e9f7d0>, 'Connection to <ip-address> timed out. (connect timeout=10)')': /api/atlas/v2/types/typedefs
[2021-02-18 18:50:17,327] {connectionpool.py:752} WARNING - Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2e9fe10>, 'Connection to <ip-address> timed out. (connect timeout=10)')': /api/atlas/v2/types/typedefs
[2021-02-18 18:50:27,338] {taskinstance.py:1150} ERROR - HTTPConnectionPool(host='<ip-address>, port=21000): Max retries exceeded with url: /api/atlas/v2/types/typedefs (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb1e2ea3490>, 'Connection to <ip-address> timed out. (connect timeout=10)'))

编辑：按要求发布代码

气流.cfg 配置

backend = airflow.lineage.backend.atlas.AtlasBackend

[atlas]
host = <ip-address>
port = 21000
username = admin
password = <password>
4

1 回答 1

0

我能够通过将 IP 地址设置为私有 IP 地址而不是运行 atlas 的 ec2 的公共 IP 地址来解决该问题。另外，我不得不更新运行 apache atlas 的 ec2 的安全组入站规则，以允许气流 webserver 流量的私有 ip 地址进入。

于 2021-02-24T15:28:27.773 回答