尝试将question-generation
t5 模型转换为torchscript
模型,同时执行此错误
ValueError:您必须指定decoder_input_ids 或decoder_inputs_embeds
这是我在 colab 上运行的代码。
!pip install -U transformers==3.0.0
!python -m nltk.downloader punkt
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch
model = AutoModelForSeq2SeqLM.from_pretrained('valhalla/t5-base-qg-hl')
t_input = 'Python is a programming language. It is developed by <hl> Guido Van Rossum <hl>. </s>'
tokenizer = AutoTokenizer.from_pretrained('valhalla/t5-base-qg-hl', return_tensors = 'pt')
def _tokenize(
inputs,
padding=True,
truncation=True,
add_special_tokens=True,
max_length=64
):
inputs = tokenizer.batch_encode_plus(
inputs,
max_length=max_length,
add_special_tokens=add_special_tokens,
truncation=truncation,
padding="max_length" if padding else False,
pad_to_max_length=padding,
return_tensors="pt"
)
return inputs
token = _tokenize(t_input, padding=True, truncation=True)
traced_model = torch.jit.trace(model, [token['input_ids'], token['attention_mask']] )
torch.jit.save(traced_model, "traced_t5.pt")
得到这个错误
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-f9b449524ef1> in <module>()
32
33
---> 34 traced_model = torch.jit.trace(model, [token['input_ids'], token['attention_mask']] )
35 torch.jit.save(traced_model, "traced_t5.pt")
7 frames
/usr/local/lib/python3.6/dist-packages/transformers/modeling_t5.py in forward(self, input_ids, attention_mask, encoder_hidden_states, encoder_attention_mask, inputs_embeds, head_mask, past_key_value_states, use_cache, output_attentions, output_hidden_states)
682 else:
683 if self.is_decoder:
--> 684 raise ValueError("You have to specify either decoder_input_ids or decoder_inputs_embeds")
685 else:
686 raise ValueError("You have to specify either input_ids or inputs_embeds")
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds
如何解决这个问题?或者有没有更好的方法将 t5 模型转换为torchscript
.
谢谢你。