0

我正在尝试微调 Huggingface bert-large-uncased-whole-word-masking 模型，并且在训练时出现这样的类型错误：

“TypeError：只有单个元素的整数张量可以转换为索引”

这是代码：


train_inputs = tokenizer(text_list[0:457], return_tensors='pt', max_length=512, truncation=True, padding='max_length')
train_inputs['labels']= train_inputs.input_ids.detach().clone()

然后我随机屏蔽了 input-ids 中大约 15% 的单词，并为数据集定义了一个类，然后错误发生在训练循环中：

class MeditationsDataset(torch.utils.data.Dataset):
    def __init__(self, encodings):
        self.encodings= encodings
    def __getitem__(self, idx):
        return {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
    def __len__(self):
        return self.encodings.input_ids

train_dataset = MeditationsDataset(train_inputs)
train_dataloader = torch.utils.data.DataLoader(dataset= train_dataset, batch_size=8, shuffle=False)

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

from transformers import BertModel, AdamW

model = BertModel.from_pretrained("bert-large-uncased-whole-word-masking")
model.to(device)
model.train()

optim = AdamW(model.parameters(), lr=1e-5)
num_epochs = 2
from tqdm.auto import tqdm

for epoch in range(num_epochs):
    loop = tqdm(train_dataloader, leave=True)
    for batch in loop:
        batch = {k: v.to(device) for k, v in batch.items()}
        outputs = model(**batch)
        loss = outputs.loss
        loss.backward()

        optimizer.step()
        optimizer.zero_grad()

错误发生在“for batch in loop”中

有人理解它并知道如何解决这个问题吗？在此先感谢您的帮助

4

0 回答 0