1

我找不到在没有兼容性问题的情况下 pip 安装以下 Python 模块的方法(来自 Red Hat Enterprise Linux 版本 8.2 上的 requirements.txt 文件):

tensorflow_gpu
tensorflow_addons
tensorflow_io
pandas
pillow
matplotlib

我试图指定版本,如

tensorflow_gpu
tensorflow_addons==2.6.0
tensorflow_io
pandas
pillow
matplotlib

但是,安装总是失败。

我在虚拟环境中尝试了上述方法。在我最近的尝试中,我删除了站点包以避免进一步的兼容性问题:

~$ mkdir ./venv
~$ python3 -m venv ./venv
~$ source ./venv/bin/activate
(venv) ~$ cd ./venv
(venv) ./venv$ pip3.8 freeze | xargs pip3.8 uninstall -y
(venv) ./venv$ pip3.8 install -r requirements.txt

安装完成后出现问题,我无法像这样导入 tensorflow 和 tensorflow_addons:

(venv) ./venv$ python3
>>> import tensorflow as tf
>>> import tensorflow_addons as tfa

由于以下错误(完整输出进一步向下):

tensorflow.python.framework.errors_impl.AlreadyExistsError: Another metric with the same name already exists

是否有可以找到匹配版本的地方,或者仅通过指定模块的某些版本无法解决问题?


完整模块导入错误输出:

2022-02-09 06:29:07.367472: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/api/keras/optimizers
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow_addons/__init__.py", line 21, in <module>
    from tensorflow_addons import activations
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow_addons/activations/__init__.py", line 17, in <module>
    from tensorflow_addons.activations.gelu import gelu
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow_addons/activations/gelu.py", line 19, in <module>
    from tensorflow_addons.utils.types import TensorLike
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow_addons/utils/types.py", line 24, in <module>
    from keras.engine import keras_tensor
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/__init__.py", line 25, in <module>
    from keras import models
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/models.py", line 20, in <module>
    from keras import metrics as metrics_module
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/metrics.py", line 26, in <module>
    from keras import activations
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/activations.py", line 20, in <module>
    from keras.layers import advanced_activations
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/layers/__init__.py", line 23, in <module>
    from keras.engine.input_layer import Input
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/engine/input_layer.py", line 21, in <module>
    from keras.engine import base_layer
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/engine/base_layer.py", line 43, in <module>
    from keras.mixed_precision import loss_scale_optimizer
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/mixed_precision/loss_scale_optimizer.py", line 18, in <module>
    from keras import optimizers
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/optimizers.py", line 26, in <module>
    from keras.optimizer_v2 import adadelta as adadelta_v2
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/optimizer_v2/adadelta.py", line 22, in <module>
    from keras.optimizer_v2 import optimizer_v2
  File "/opt/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages/keras/optimizer_v2/optimizer_v2.py", line 36, in <module>
    keras_optimizers_gauge = tf.__internal__.monitoring.BoolGauge(
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow/python/eager/monitoring.py", line 360, in __init__
    super(BoolGauge, self).__init__('BoolGauge', _bool_gauge_methods,
  File "/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages/tensorflow/python/eager/monitoring.py", line 135, in __init__
    self._metric = self._metric_methods[self._label_length].create(*args)
tensorflow.python.framework.errors_impl.AlreadyExistsError: Another metric with the same name already exists.

模块安装期间 这是 requirements.txt 中模块安装期间的输出:

WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Requirement already satisfied: tensorflow==2.6.0 in /pfs/data5/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (2.6.0)
Collecting tensorflow_gpu==2.6.0
  Using cached tensorflow_gpu-2.6.0-cp38-cp38-manylinux2010_x86_64.whl (458.4 MB)
Requirement already satisfied: tensorflow_addons in /pfs/data5/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (0.15.0)
Requirement already satisfied: tensorflow_io in /pfs/data5/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (0.21.0)
Requirement already satisfied: pandas in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (1.4.0)
Requirement already satisfied: pillow in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (9.0.0)
Requirement already satisfied: matplotlib in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from -r requirements.txt (line 7)) (3.5.1)
Requirement already satisfied: wheel~=0.35 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.37.1)
Collecting absl-py~=0.10
  Using cached absl_py-0.15.0-py3-none-any.whl (132 kB)
Requirement already satisfied: google-pasta~=0.2 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.2.0)
Collecting six~=1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting flatbuffers~=1.12.0
  Using cached flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.1.2)
Requirement already satisfied: tensorflow-estimator~=2.6 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2.7.0)
Collecting typing-extensions~=3.7.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Requirement already satisfied: keras~=2.6 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2.7.0)
Requirement already satisfied: termcolor~=1.1.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.1.0)
Collecting h5py~=3.1.0
  Using cached h5py-3.1.0-cp38-cp38-manylinux1_x86_64.whl (4.4 MB)
Collecting numpy~=1.19.2
  Using cached numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
Requirement already satisfied: gast==0.4.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.43.0)
Requirement already satisfied: opt-einsum~=3.3.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.3.0)
Requirement already satisfied: tensorboard~=2.6 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (2.8.0)
Collecting clang~=5.0
  Using cached clang-5.0-py3-none-any.whl
Collecting wrapt~=1.12.1
  Using cached wrapt-1.12.1-cp38-cp38-linux_x86_64.whl
Requirement already satisfied: protobuf>=3.9.2 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (3.19.4)
Requirement already satisfied: astunparse~=1.6.3 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorflow==2.6.0->-r requirements.txt (line 1)) (1.6.3)
Requirement already satisfied: typeguard>=2.7 in /pfs/data5/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages (from tensorflow_addons->-r requirements.txt (line 3)) (2.13.3)
Requirement already satisfied: tensorflow-io-gcs-filesystem==0.21.0 in /pfs/data5/home/kit/ifgg/mp3890/.local/lib/python3.8/site-packages (from tensorflow_io->-r requirements.txt (line 4)) (0.21.0)
Requirement already satisfied: pytz>=2020.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from pandas->-r requirements.txt (line 5)) (2021.3)
Requirement already satisfied: python-dateutil>=2.8.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from pandas->-r requirements.txt (line 5)) (2.8.2)
Requirement already satisfied: fonttools>=4.22.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 7)) (4.29.0)
Requirement already satisfied: pyparsing>=2.2.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 7)) (3.0.7)
Requirement already satisfied: cycler>=0.10 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 7)) (0.11.0)
Requirement already satisfied: packaging>=20.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 7)) (21.3)
Requirement already satisfied: kiwisolver>=1.0.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from matplotlib->-r requirements.txt (line 7)) (1.3.2)
Requirement already satisfied: setuptools>=41.0.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (41.6.0)
Requirement already satisfied: requests<3,>=2.21.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (2.27.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (1.8.1)
Requirement already satisfied: markdown>=2.6.8 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (3.3.6)
Requirement already satisfied: google-auth<3,>=1.6.3 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (2.5.0)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (0.6.1)
Requirement already satisfied: werkzeug>=0.11.15 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (2.0.2)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (0.4.6)
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (5.0.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (4.10.1)
Requirement already satisfied: idna<4,>=2.5 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (1.26.8)
Requirement already satisfied: certifi>=2017.4.17 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (2.0.11)
Requirement already satisfied: zipp>=0.5 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (3.7.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (0.4.8)
Requirement already satisfied: oauthlib>=3.0.0 in /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.6.0->-r requirements.txt (line 1)) (3.2.0)
Installing collected packages: wrapt, typing-extensions, flatbuffers, clang, six, numpy, h5py, absl-py, tensorflow_gpu
  Attempting uninstall: wrapt
    Found existing installation: wrapt 1.13.3
    Not uninstalling wrapt at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'wrapt'. No files were found to uninstall.
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.0.1
    Not uninstalling typing-extensions at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'typing_extensions'. No files were found to uninstall.
  Attempting uninstall: flatbuffers
    Found existing installation: flatbuffers 2.0
    Not uninstalling flatbuffers at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'flatbuffers'. No files were found to uninstall.
  Attempting uninstall: six
    Found existing installation: six 1.16.0
    Not uninstalling six at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'six'. No files were found to uninstall.
  Attempting uninstall: numpy
    Found existing installation: numpy 1.22.1
    Not uninstalling numpy at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'numpy'. No files were found to uninstall.
  Attempting uninstall: h5py
    Found existing installation: h5py 3.6.0
    Not uninstalling h5py at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'h5py'. No files were found to uninstall.
  Attempting uninstall: absl-py
    Found existing installation: absl-py 1.0.0
    Not uninstalling absl-py at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'absl-py'. No files were found to uninstall.
  Attempting uninstall: tensorflow_gpu
    Found existing installation: tensorflow-gpu 2.7.0
    Not uninstalling tensorflow-gpu at /pfs/data5/software_uc2/bwhpc/common/jupyter/tensorflow/2022-01-11/lib/python3.8/site-packages, outside environment /pfs/data5/home/kit/ifgg/mp3890/ven0
    Can't uninstall 'tensorflow-gpu'. No files were found to uninstall.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
bokeh 2.4.2 requires typing-extensions>=3.10.0, but you have typing-extensions 3.7.4.3 which is incompatible.
black 22.1.0 requires typing-extensions>=3.10.0.0; python_version < "3.10", but you have typing-extensions 3.7.4.3 which is incompatible.
Successfully installed absl-py-0.15.0 clang-5.0 flatbuffers-1.12 h5py-3.1.0 numpy-1.19.5 six-1.15.0 tensorflow_gpu-2.6.0 typing-extensions-3.7.4.3 wrapt-1.12.1
WARNING: You are using pip version 22.0.2; however, version 22.0.3 is available.
You should consider upgrading via the '/pfs/data5/home/kit/ifgg/mp3890/ven0/bin/python3 -m pip install --upgrade pip' command.

按照使用 -m pip (as python3 -m pip install -r requirements.txt) 调用 Python 的建议导致相同的输出(该消息除外)。

4

1 回答 1

0

该问题是由后台加载的 jupyter/tensorflow 引起的。以下解决了这个问题:

module purge
module load devel/cuda
/usr/bin/python3.8 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

非常感谢我们集群支持团队的 Samuel Braun

于 2022-02-09T08:11:10.077 回答