1

我已经在 AWS 上使用 Docker 安装了 Airflow 2.0，并尝试使用 DockerOperator 提取我自己的私有 docker 映像 GitLab 容器注册表，但被拒绝许可。

如何从 GitLab 中提取我的 Docker 映像以及如何为 Docker 映像 GitLab 注册表创建连接？

dag 文件链接 Docker yml 文件链接

我的 dag 代码：

java_test_scheduler = DockerOperator(
    task_id='Java-hello-world-test-scheduler',
    image='registry.gitlab.com/mobinalhassan/jamay_aeronova:latest',
    auto_remove=True,
    force_pull=True,
    dag=dag
)

我还尝试通过添加 docker 卷：

- /var/run/docker.sock:/var/run/docker.sock
- $HOME/.ssh:/root/.ssh:ro
- $HOME/.docker:/root/.docker

错误：

sock.connect(self.unix_socket) PermissionError: [Errno 13] Permission denied
4

2 回答 2

2
  1. I have tried without docker you can check this answer for details on that
  2. I wanted to keep least dependency on airflow and hence didn't create the connection from airflow whose details are mentioned in the above referenced answer link
  3. As far as your dag link is concerned, I had tried similar approach but could not make it work with image='registry.gitlab.com/mobinalhassan/jamay_aeronova:latest', however it did work out when I removed the :latest and use it like image='registry.gitlab.com/mobinalhassan/jamay_aeronova',, try this out and let me know if it works
  4. For full details on pulling private docker image, I have jotted down a step by step tutorial which you can refer at - this link
于 2021-05-04T08:59:11.913 回答
0

从私有 Docker 注册表中提取时，您的 dag 缺少 dag_conn_id 定义：

If a login to a private registry is required prior to pulling the image, a Docker connection needs to be configured in Airflow and the connection ID be provided with the parameter docker_conn_id.

https://airflow.apache.org/docs/apache-airflow-providers-docker/stable/_api/airflow/providers/docker/operators/docker/index.html

于 2021-05-03T09:33:46.110 回答