我正在尝试使用 BERT 在 Python 中进行命名实体识别,并使用pip install transformers
. 然后,当我尝试运行此代码时:
import torch
from torch.utils.data import TensorDataset, DataLoader, RandomSampler, SequentialSampler
from transformers import BertTokenizer, BertConfig
from keras.preprocessing.sequence import pad_sequences
from sklearn.model_selection import train_test_split
torch.__version__
我收到此错误:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so, 2): Symbol not found: ____chkstk_darwin
Referenced from: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so (which was built for Mac OS X 10.15)
Expected in: /usr/lib/libSystem.B.dylib
in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/tokenizers/tokenizers.cpython-38-darwin.so
错误发生在这一行:from transformers import BertTokenizer, BertConfig
但我不知道如何解决这个问题。