8

我正在使用torch.save()和torch.load()命令保存和加载模型。

在 Docker Container 中加载经过微调的简单变压器模型时，我遇到了这个我无法解决的错误：

 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 594, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.7/dist-packages/torch/serialization.py", line 853, in _load
    result = unpickler.load()
  File "/usr/local/lib/python3.7/dist-packages/transformers/models/xlm_roberta/tokenization_xlm_roberta.py", line 161, in __setstate__
    self.sp_model.Load(self.vocab_file)
  File "/usr/local/lib/python3.7/dist-packages/sentencepiece.py", line 367, in Load
    return self.LoadFromFile(model_file)
  File "/usr/local/lib/python3.7/dist-packages/sentencepiece.py", line 177, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
OSError: Not found: "/home/jupyter/.cache/huggingface/transformers/9df9ae4442348b73950203b63d1b8ed2d18eba68921872aee0c3a9d05b9673c6.00628a9eeb8baf4080d44a0abe9fe8057893de20c7cb6e6423cddbf452f7d4d8": No such file or directory Error #2

如果有人对此有任何想法，请告诉我。

我在用：

  • 火炬==1.7.1+cu101
  • 句子转换器 0.3.9
  • 简单变压器 0.51.15
  • 变压器 4.4.2
  • 张量流 2.2.0
4

1 回答 1

1

我建议使用state_dict对象——Python 字典，因为它们可以很容易地保存、更新和恢复，让您可以灵活地稍后恢复模型。以下是用于保存模型的推荐保存/加载方法state_dict：

节省

torch.save(model.state_dict(), PATH)

加载

model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH))
model.eval()
于 2021-04-11T12:07:52.357 回答