amazon-web-services - AWS Batch Arrays - 数组大小？

Question

我正在使用 AWS Batch 并已开始使用数组作业。 AWS_BATCH_JOB_ARRAY_INDEX作为环境变量传递给容器。

数组大小是否以某种方式传递？必须知道该指数是否与 5 个工作或 1000 个工作相关。目前我将它作为我自己的环境变量传递，但认为该信息已经以某种方式传递给容器。

Answer 1

This is not possible at the moment. I've made a feature request for it, which you can upvote here: https://github.com/aws/containers-roadmap/issues/1631

In the meantime, I found a hacky workaround. The job ID for array workers appears to conform to $PARENT_JOB_ID:$AWS_BATCH_JOB_ARRAY_INDEX. So, to the extent that you can rely on this formatting of array worker IDs, you can describe the parent job and get the total array size from there. Here's an example using boto3:

import os
import boto3

worker_job_id = os.environ['AWS_BATCH_JOB_ID']
parent_job_id = worker_job_id.split(":")[0]

response = boto3.client('batch').describe_jobs(jobs=[parent_job_id])

parent_job = response['jobs'][0]
array_size = parent_job.get('arrayProperties', {}).get("size")

print("array_size =", array_size)

Answer 2

如果我的理解是正确的，你是在问数组大小应该在 aws 批处理中传递到哪里？

在 Jobs 部分，单击 submit job - 在 environment 中选择 Array。

参考：https ://docs.aws.amazon.com/batch/latest/userguide/submit_job.html