0

我尝试使用 GilBERTo（tokenizer 和模型）和 RoBERTa 的意大利语版本（来自 CamemBert）训练 NER 模型，但我有以下错误，有人可以帮助我理解它，也许是一个可能的解决方案？

该解决方案复制了：GitHub：R-BERT 实体关系 （我只是将 utils.py 更改为：

def load_tokenizer(args):
    tokenizer = AutoTokenizer.from_pretrained("idb-ita/gilberto-uncased-from-camembert", do_lower_case=True)
    #tokenizer = MODEL_CLASSES[args.model_type][2].from_pretrained(args.model_name_or_path)
    tokenizer.add_special_tokens({"additional_special_tokens": ADDITIONAL_SPECIAL_TOKENS})
    return tokenizer

) 然后我在 main.py 中使用模型“roberta”，只需将 Tokenizer 更改为 GilBERTo。我使用了与原始项目相同格式的不同数据集。注意：由于缓存上传的大小问题，我禁用了 GPU，并且我使用了变形金刚 2.4.1，非常感谢！

03/04/2020 16:33:01 - INFO - trainer -   ***** Config loaded *****
03/04/2020 16:33:01 - INFO - transformers.modeling_utils -   loading weights file ./model\pytorch_model.bin
03/04/2020 16:33:01 - INFO - transformers.modeling_utils -   loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/roberta-base-pytorch_model.bin from cache at C:\Users\ADMIN\.cache\torch\transformers\228756ed15b6d200d7cb45aaef08c087e2706f54cb912863d2efe07c89584eb7.49b88ba7ec2c26a7558dda98ca3884c3b80fa31cf43a1b1f23aef3ff81ba344e
Traceback (most recent call last):
  File "C:\Users\ADMIN\R-BERT\trainer.py", line 196, in load_model
    self.model = self.model_class.from_pretrained(self.args.model_dir, config=self.bert_config, args=self.args)
  File "C:\Users\ADMIN\Miniconda3\envs\envTransformers\lib\site-packages\transformers\modeling_utils.py", line 463, in from_pretrained
    model = cls(config, *model_args, **model_kwargs)
  File "C:\Users\ADMIN\R-BERT\model.py", line 31, in __init__
    self.bert = PRETRAINED_MODEL_MAP[args.model_type].from_pretrained(args.model_name_or_path, config=bert_config)  # Load pretrained bert
  File "C:\Users\ADMIN\Miniconda3\envs\envTransformers\lib\site-packages\transformers\modeling_utils.py", line 555, in from_pretrained
    model.__class__.__name__, "\n\t".join(error_msgs)
RuntimeError: Error(s) in loading state_dict for RobertaModel:
        size mismatch for roberta.embeddings.word_embeddings.weight: copying a param with shape torch.Size([50265, 768]) from checkpoint, the shape in current model is torch.Size([30522, 768]).
        size mismatch for roberta.embeddings.position_embeddings.weight: copying a param with shape torch.Size([514, 768]) from checkpoint, the shape in current model is torch.Size([512, 768]).
        size mismatch for roberta.embeddings.token_type_embeddings.weight: copying a param with shape torch.Size([1, 768]) from checkpoint, the shape in current model is torch.Size([2, 768]).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 60, in <module>
    trainer.load_model()
  File "C:\Users\ADMIN\R-BERT\trainer.py", line 200, in load_model
    raise Exception("Some model files might be missing...")
Exception: Some model files might be missing...
4

0 回答 0