我有一个 Cloud Build 触发器,可将文件从我的 Cloud Source Repo 移动到 Cloud Composer GCS Bucker。
我使用以下 YAML 文件作为起点。
steps:
- name: ubuntu
args: ['bash', '-c', "echo '$COMMIT_SHA' > REVISION.txt"]
- name: gcr.io/cloud-builders/gsutil
args:
- '-m'
- 'rsync'
- '-d'
- '-r'
- ''
- 'gs://${_GCS_BUCKET}/dags'
但是,我的 Git 结构如下所示:
.
| project_strcuture.png
| log.txt
| README.md
| cloudbuild.yaml
| requirments.txt
|
+---PROJECT_A
| Project_A_PROD.py
| Project_A - flow chart.png
| README.md
|
+---Project_B_APAC
| +---Project_B1_APAC
| | Project_B1_APAC_flow_chart.png
| | Project_B1_APAC.py
| | README.md
| |
| +---Project_B2_APAC
| | Project_B2_APAC.py
| | Project_B2_APAC_flow_chart.png
| | README.md
| |
| \---Project_B3_APAC
| Project_B3_APAC.py
| Project_B3_APAC_flow_chart.png
| README.md
|
\---Global_Pipeline
| \---API_BQ_Data_Validation
| | compare.csv
| | GlobalBQ_comaprison.py
| | GlobalBQ_comaprison.ipynb
我只希望特定的 python 文件(有效的 Cloud Composer DAG)从我的 Git 移动到 Cloud Composer。有什么办法吗?