2

我有一个 Cloud Build 触发器,可将文件从我的 Cloud Source Repo 移动到 Cloud Composer GCS Bucker。

我使用以下 YAML 文件作为起点。

steps:
- name: ubuntu
  args: ['bash', '-c', "echo '$COMMIT_SHA' > REVISION.txt"]
- name: gcr.io/cloud-builders/gsutil
  args:
    - '-m'
    - 'rsync'
    - '-d'
    - '-r'
    - ''
    - 'gs://${_GCS_BUCKET}/dags'

但是,我的 Git 结构如下所示:

.
|   project_strcuture.png
|   log.txt
|   README.md
|   cloudbuild.yaml
|   requirments.txt
|              
+---PROJECT_A
|       Project_A_PROD.py
|       Project_A - flow chart.png
|       README.md
|                      
+---Project_B_APAC
|   +---Project_B1_APAC
|   |       Project_B1_APAC_flow_chart.png
|   |       Project_B1_APAC.py
|   |       README.md
|   |       
|   +---Project_B2_APAC
|   |       Project_B2_APAC.py
|   |       Project_B2_APAC_flow_chart.png
|   |       README.md
|   |       
|   \---Project_B3_APAC
|           Project_B3_APAC.py
|           Project_B3_APAC_flow_chart.png
|           README.md
|
\---Global_Pipeline
|   \---API_BQ_Data_Validation
|   |       compare.csv
|   |       GlobalBQ_comaprison.py
|   |       GlobalBQ_comaprison.ipynb

我只希望特定的 python 文件(有效的 Cloud Composer DAG)从我的 Git 移动到 Cloud Composer。有什么办法吗?

4

0 回答 0