我正在关注链接：https : //kubernetes.dask.org/en/latest/，在 Kubernetes 集群上运行 dask 数组。
脚步：
- 在 3 个节点（1 个主节点和 2 个工作节点）上安装了 Kubernetes。
- 安装 miniconda3
- pip install dask-kubernetes
- dask_example.py 带有运行 dask 数组的代码（与链接上给出的示例相同）
- 带有 pod 配置的 Worker-spec.yml 文件（与链接上给出的示例相同）
运行示例代码时，worker pod 显示错误状态如下：
(base) [root@k8s-master example]# kubectl logs workerpod
+ '[' '' ']'
+ '[' -e /opt/app/environment.yml ']'
+ echo 'no environment.yml'
no environment.yml
+ '[' '' ']'
+ '[' 'fastparquet git+https://github.com/dask/distributed' ']'
+ echo 'EXTRA_PIP_PACKAGES environment variable found. Installing.'
+ /opt/conda/bin/pip install fastparquet git+https://github.com/dask/distributed
EXTRA_PIP_PACKAGES environment variable found. Installing.
Collecting git+https://github.com/dask/distributed
Cloning https://github.com/dask/distributed to /tmp/pip-req-build-9pgvdhjf
Running command git clone -q https://github.com/dask/distributed /tmp/pip-req-build-9pgvdhjf
Collecting fastparquet
Downloading https://files.pythonhosted.org/packages/58/49/dccb790fa17ab3fbf84a6b848050083c7a1899e9586000e34e3e4fbf5538/fastparquet-0.3.2.tar.gz (151kB)
Requirement already satisfied: click>=6.6 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (7.0)
Requirement already satisfied: cloudpickle>=0.2.2 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (1.2.2)
Requirement already satisfied: dask>=2.7.0 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (2.8.0)
Requirement already satisfied: msgpack in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (0.6.2)
Requirement already satisfied: psutil>=5.0 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (5.6.5)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (2.1.0)
Requirement already satisfied: tblib in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (1.4.0)
Requirement already satisfied: toolz>=0.7.4 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (0.10.0)
Requirement already satisfied: tornado>=5 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (6.0.3)
Requirement already satisfied: zict>=0.1.3 in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (1.0.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.7/site-packages (from distributed==2.8.0+8.g5b33d54c) (5.1.2)
Requirement already satisfied: pandas>=0.19 in /opt/conda/lib/python3.7/site-packages (from fastparquet) (0.25.2)
Collecting numba>=0.28 (from fastparquet)
Downloading https://files.pythonhosted.org/packages/57/66/7ebc88e87b4ddf9b1c204e24d467cb0d13a7a890ecb4f83d20949f768929/numba-0.46.0-cp37-cp37m-manylinux1_x86_64.whl (3.6MB)
Requirement already satisfied: numpy>=1.11 in /opt/conda/lib/python3.7/site-packages (from fastparquet) (1.17.3)
Collecting thrift>=0.11.0 (from fastparquet)
Downloading https://files.pythonhosted.org/packages/97/1e/3284d19d7be99305eda145b8aa46b0c33244e4a496ec66440dac19f8274d/thrift-0.13.0.tar.gz (59kB)
Requirement already satisfied: six in /opt/conda/lib/python3.7/site-packages (from fastparquet) (1.12.0)
Requirement already satisfied: heapdict in /opt/conda/lib/python3.7/site-packages (from zict>=0.1.3->distributed==2.8.0+8.g5b33d54c) (1.0.1)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas>=0.19->fastparquet) (2019.3)
Requirement already satisfied: python-dateutil>=2.6.1 in /opt/conda/lib/python3.7/site-packages (from pandas>=0.19->fastparquet) (2.8.1)
Collecting llvmlite>=0.30.0dev0 (from numba>=0.28->fastparquet)
Downloading https://files.pythonhosted.org/packages/1f/3e/642ffb29ed35ca5e93f171ba327452bdee81ec76f2d711ef0f15b411928a/llvmlite-0.30.0-cp37-cp37m-manylinux1_x86_64.whl (20.2MB)
Building wheels for collected packages: fastparquet, distributed, thrift
Building wheel for fastparquet (setup.py): started
Building wheel for fastparquet (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-tcex8z1e --python-tag cp37
cwd: /tmp/pip-install-m6berq44/fastparquet/
Complete output (61 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/__init__.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/api.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/compression.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/converted_types.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/core.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/dataframe.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/encoding.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/schema.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/thrift_structures.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/util.py -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/writer.py -> build/lib.linux-x86_64-3.7/fastparquet
running egg_info
writing fastparquet.egg-info/PKG-INFO
writing dependency_links to fastparquet.egg-info/dependency_links.txt
writing requirements to fastparquet.egg-info/requires.txt
writing top-level names to fastparquet.egg-info/top_level.txt
reading manifest file 'fastparquet.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
writing manifest file 'fastparquet.egg-info/SOURCES.txt'
copying fastparquet/parquet.thrift -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/speedups.c -> build/lib.linux-x86_64-3.7/fastparquet
copying fastparquet/speedups.pyx -> build/lib.linux-x86_64-3.7/fastparquet
creating build/lib.linux-x86_64-3.7/fastparquet/benchmarks
copying fastparquet/benchmarks/columns.py -> build/lib.linux-x86_64-3.7/fastparquet/benchmarks
creating build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift
copying fastparquet/parquet_thrift/__init__.py -> build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift
creating build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift/parquet
copying fastparquet/parquet_thrift/parquet/__init__.py -> build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift/parquet
copying fastparquet/parquet_thrift/parquet/constants.py -> build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift/parquet
copying fastparquet/parquet_thrift/parquet/ttypes.py -> build/lib.linux-x86_64-3.7/fastparquet/parquet_thrift/parquet
creating build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/__init__.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_api.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_aroundtrips.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_compression.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_converted_types.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_dataframe.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_encoding.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_output.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_partition_filters_specialstrings.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_read.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_schema.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_speedups.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_thrift_structures.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_util.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/test_with_n.py -> build/lib.linux-x86_64-3.7/fastparquet/test
copying fastparquet/test/util.py -> build/lib.linux-x86_64-3.7/fastparquet/test
running build_ext
building 'fastparquet.speedups' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/fastparquet
gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.7m -I/opt/conda/lib/python3.7/site-packages/numpy/core/include -c fastparquet/speedups.c -o build/temp.linux-x86_64-3.7/fastparquet/speedups.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for fastparquet
Running setup.py clean for fastparquet
Building wheel for distributed (setup.py): started
Building wheel for distributed (setup.py): finished with status 'done'
Created wheel for distributed: filename=distributed-2.8.0+8.g5b33d54c-cp37-none-any.whl size=568764 sha256=9712974396e1221fa5dd195616e85031da70894222c2c7ff574bcfb318b5f80c
Stored in directory: /tmp/pip-ephem-wheel-cache-v5jnd4bs/wheels/aa/21/a7/d9548d684f8e074360b7ad1bd8633843dba9658288b68b3dd5
Building wheel for thrift (setup.py): started
Building wheel for thrift (setup.py): finished with status 'done'
Created wheel for thrift: filename=thrift-0.13.0-cp37-none-any.whl size=154884 sha256=c32af6aa5c4cfced68fadc2997e173f14ed0595a4bb9bb407eb7ef62794fafd8
Stored in directory: /root/.cache/pip/wheels/02/a2/46/689ccfcf40155c23edc7cdbd9de488611c8fdf49ff34b1706e
Successfully built distributed thrift
Failed to build fastparquet
Installing collected packages: llvmlite, numba, thrift, fastparquet, distributed
Running setup.py install for fastparquet: started
Running setup.py install for fastparquet: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-qrgpngpc/install-record.txt --single-version-externally-managed --compile
cwd: /tmp/pip-install-m6berq44/fastparquet/
Complete output (19 lines):
running install
running build
running build_py
running egg_info
writing fastparquet.egg-info/PKG-INFO
writing dependency_links to fastparquet.egg-info/dependency_links.txt
writing requirements to fastparquet.egg-info/requires.txt
writing top-level names to fastparquet.egg-info/top_level.txt
reading manifest file 'fastparquet.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
writing manifest file 'fastparquet.egg-info/SOURCES.txt'
running build_ext
building 'fastparquet.speedups' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/fastparquet
gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.7m -I/opt/conda/lib/python3.7/site-packages/numpy/core/include -c fastparquet/speedups.c -o build/temp.linux-x86_64-3.7/fastparquet/speedups.o
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m6berq44/fastparquet/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-qrgpngpc/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
+ exec dask-worker --nthreads 2 --no-bokeh --memory-limit 6GB --death-timeout 60
/opt/conda/lib/python3.7/site-packages/distributed/cli/dask_worker.py:252: UserWarning: The --bokeh/--no-bokeh flag has been renamed to --dashboard/--no-dashboard.
"The --bokeh/--no-bokeh flag has been renamed to --dashboard/--no-dashboard. "
distributed.nanny - INFO - Start Nanny at: 'tcp://10.44.0.1:46597'
distributed.worker - INFO - Start worker at: tcp://10.44.0.1:36897
distributed.worker - INFO - Listening to: tcp://10.44.0.1:36897
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Threads: 2
distributed.worker - INFO - Memory: 6.00 GB
distributed.worker - INFO - Local Directory: /worker-vlqvrk15
distributed.worker - INFO - -------------------------------------------------
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.worker - INFO - Waiting to connect to: tcp://172.16.0.76:42143
distributed.nanny - INFO - Closing Nanny at 'tcp://10.44.0.1:46597'
distributed.worker - INFO - Stopping worker at tcp://10.44.0.1:36897
distributed.worker - INFO - Closed worker has not yet started: None
distributed.dask_worker - INFO - Timed out starting worker
distributed.dask_worker - INFO - End worker