0

我正在尝试在 kubernetes 上启动并运行 dask。下面实际上是 dask-kubernetes 的一个hello world，但我遇到了下面的错误。

主要.py：

import os
from dask_kubernetes import KubeCluster
from dask.distributed import Client
import dask.array as da


if __name__ == '__main__':
    path_to_src = os.path.dirname(os.path.abspath(__file__))
    cluster = KubeCluster(os.path.join(path_to_src, 'pod-spec.yaml'), namespace='124381-dev')
    print('Cluster constructed')

    cluster.scale(10)
    # print('Cluster scaled')

    # Connect Dask to the cluster
    client = Client(cluster)
    print('Client constructed')

    # Create a large array and calculate the mean
    array = da.ones((100, 100, 100))
    print('Created big array')
    print(array.mean().compute())  # Should print 1.0
    print('Computed mean')

输出：

$ python src/main.py 
Creating scheduler pod on cluster. This may take some time.
Forwarding from 127.0.0.1:60611 -> 8786
Handling connection for 60611
Handling connection for 60611
Handling connection for 60611
Cluster constructed
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f1f874b8130>>, <Task finished name='Task-54' coro=<SpecCluster._correct_state_internal() done, defined at /home/cliff/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/deploy/spec.py:327> exception=TypeError("unsupported operand type(s) for +=: 'NoneType' and 'list'")>)
Traceback (most recent call last):
  File "/home/cliff/anaconda3/envs/dask/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/cliff/anaconda3/envs/dask/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/cliff/anaconda3/envs/dask/lib/python3.8/site-packages/distributed/deploy/spec.py", line 358, in _correct_state_internal
    worker = cls(self.scheduler.address, **opts)
  File "/home/cliff/anaconda3/envs/dask/lib/python3.8/site-packages/dask_kubernetes/core.py", line 151, in __init__
    self.pod_template.spec.containers[0].args += worker_name_args
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'list'
Handling connection for 60611
Handling connection for 60611
Client constructed
Created big array

请注意，输出末尾没有终端提示 - 它仍在运行但永远不会进行。在另一个终端kubectl get pods中也显示“悬崖测试”正在运行。

吊舱规格.yaml：

apiVersion: v1
kind: Pod
metadata:
  name: cliff-testing
  labels:
    app: cliff-docker-test
spec:
  imagePullSecrets:
  - name: <redacted>
  securityContext:
    runAsUser: 1000
  restartPolicy: OnFailure
  containers:
  - name: cliff-test-container
    image: <redacted: works with docker pull>
    imagePullPolicy: Always
    resources:
      limits:
        cpu: 2
        memory: 4G
      requests:
        cpu: 1
        memory: 2G
4

1 回答 1

0

在 pod 模板 (pod-spec.yaml) 中，metadata.name设置了该字段。删除它允许代码运行。dask-kubernetes 似乎创建了一个名为“dask-<user>-<random>”的调度程序 pod，并对工作人员遵循相同的命名方法。通过修复 pod 模板中的名称，dask-kubernetes 试图创建与调度程序 pod（以及彼此）同名的工作 pod，这是非法的。

如果您想以不同的方式命名 pod，可以name在构造时使用关键字参数KubeCluster来命名 pod（dask 会自动为每个 pod 附加一个随机字符串到此名称）。

例如，下面的示例将导致每个 pod（调度程序和工作程序）被命名为“my-dask-pods-<random>”

from dask_kubernetes import KubeCluster
cluster = KubeCluster('pod-spec.yaml', name='my-dask-pods-')
于 2021-08-24T20:33:30.180 回答