1

我正在尝试从他们在 google colab 上的网站执行 simpletransformers 示例。

例子：

from simpletransformers.classification import ClassificationModel, ClassificationArgs
import pandas as pd
import logging


logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)

# Preparing train data
train_data = [
    ["Aragorn was the heir of Isildur", 1],
    ["Frodo was the heir of Isildur", 0],
]
train_df = pd.DataFrame(train_data)
train_df.columns = ["text", "labels"]

# Preparing eval data
eval_data = [
    ["Theoden was the king of Rohan", 1],
    ["Merry was the king of Rohan", 0],
]
eval_df = pd.DataFrame(eval_data)
eval_df.columns = ["text", "labels"]

# Optional model configuration
model_args = ClassificationArgs(num_train_epochs=1)

# Create a ClassificationModel
model = ClassificationModel(
    "roberta", "roberta-base", args=model_args
)

# Train the model
model.train_model(train_df)

# Evaluate the model
result, model_outputs, wrong_predictions = model.eval_model(eval_df)

# Make predictions with the model
predictions, raw_outputs = model.predict(["Sam was a Wizard"])

但它给了我以下错误：

VersionConflict：tokenizers==0.9.4 是该模块正常运行所必需的，但发现 tokenizers==0.10.0。尝试： pip install tr​​ansformers -U 或 pip install -e '.[dev]' 如果你正在使用 git master

我已经尝试过!pip install transformers -U，甚至!pip install tokenizers==0.9.4 仍然出现同样的错误。我之前已经执行过这段代码，它的工作很有趣，但现在它给出了提到的错误。

4

2 回答 2

3

我把这个放在这里以防有人面临同样的问题。我得到了创作者本人的帮助。

Workaround:
Install tokenizers==0.9.4 before install simpletransformers

In Colab for example;

!pip install tokenizers==0.9.4
!pip install simpletransformers

https://github.com/ThilinaRajapakse/simpletransformers/issues/950

于 2021-01-29T13:12:36.257 回答
0

如上所述，我认为您基本上只需要!pip install tokenizers==0.9.4卸载tokenizers-0.10.0和安装tokenizers-0.9.4.

于 2022-01-12T14:42:38.020 回答