0

问题

我正在尝试运行 spaCy CLI,但是当我运行调试时,我的训练数据和开发数据似乎不正确:

| => python3 -m spacy debug-data en 

./CLI_train_randsplit_anno191022.json ./CLI_dev_randsplit_anno191022.json --pipeline ner --verbose 




=========================== Data format validation =========================== 

✔ Corpus is loadable 




=============================== Training stats =============================== 

Training pipeline: ner 

Starting with blank model 'en' 

0 training docs 

0 evaluation docs 

✔ No overlap between training and evaluation data 

✘ Low number of examples to train from a blank model (0) 

It's recommended to use at least 2000 examples (minimum 100) 




============================== Vocab & Vectors ============================== 

ℹ 0 total words in the data (0 unique) 

10 most common words: 

ℹ No word vectors present in the model 




========================== Named Entity Recognition ========================== 

ℹ 0 new labels, 0 existing labels 

0 missing values (tokens with '-' label) 

✔ Good amount of examples for all labels 

✔ Examples without occurrences available for all labels 

✔ No entities consisting of or starting/ending with whitespace 




================================== Summary ================================== 

✔ 5 checks passed 

✘ 1 error 

无论如何尝试训练会产生:

| => python3 -m spacy train en ./models/CLI_1 ./CLI_train_randsplit_anno191022.json ./CLI_dev_randsplit_anno191022.json -n 150 -p 'ner' --verbose 

dropout_from = 0.2 by default 

dropout_to = 0.2 by default 

dropout_decay = 0.0 by default 

batch_from = 100.0 by default 

batch_to = 1000.0 by default 

batch_compound = 1.001 by default 

Training pipeline: ['ner'] 

Starting with blank model 'en' 

beam_width = 1 by default 

beam_density = 0.0 by default 

beam_update_prob = 1.0 by default 

Counting training words (limit=0) 

learn_rate = 0.001 by default 

optimizer_B1 = 0.9 by default 

optimizer_B2 = 0.999 by default 

optimizer_eps = 1e-08 by default 

L2_penalty = 1e-06 by default 

grad_norm_clip = 1.0 by default 

parser_hidden_depth = 1 by default 

subword_features = True by default 

conv_depth = 4 by default 

bilstm_depth = 0 by default 

parser_maxout_pieces = 2 by default 

token_vector_width = 96 by default 

hidden_width = 64 by default 

embed_size = 2000 by default 




Itn  NER Loss   NER P   NER R   NER F   Token %  CPU WPS 

---  ---------  ------  ------  ------  -------  ------- 

✔ Saved model to output directory 

models/CLI_1/model-final 




Traceback (most recent call last): 

  File "/usr/local/lib/python3.7/site-packages/spacy/cli/train.py", line 389, in train 

    scorer = nlp_loaded.evaluate(dev_docs, verbose=verbose) 

  File "/usr/local/lib/python3.7/site-packages/spacy/language.py", line 673, in evaluate 

    docs, golds = zip(*docs_golds) 

ValueError: not enough values to unpack (expected 2, got 0) 




During handling of the above exception, another exception occurred: 




Traceback (most recent call last): 

  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main 

    "__main__", mod_spec) 

  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code 

    exec(code, run_globals) 

  File "/usr/local/lib/python3.7/site-packages/spacy/__main__.py", line 35, in <module> 

    plac.call(commands[command], sys.argv[1:]) 

  File "/usr/local/lib/python3.7/site-packages/plac_core.py", line 328, in call 

    cmd, result = parser.consume(arglist) 

  File "/usr/local/lib/python3.7/site-packages/plac_core.py", line 207, in consume 

    return cmd, self.func(*(args + varargs + extraopts), **kwargs) 

  File "/usr/local/lib/python3.7/site-packages/spacy/cli/train.py", line 486, in train 

    best_model_path = _collate_best_model(meta, output_path, nlp.pipe_names) 

  File "/usr/local/lib/python3.7/site-packages/spacy/cli/train.py", line 548, in _collate_best_model 

    bests[component] = _find_best(output_path, component) 

  File "/usr/local/lib/python3.7/site-packages/spacy/cli/train.py", line 567, in _find_best 

    accs = srsly.read_json(epoch_model / "accuracy.json") 

  File "/usr/local/lib/python3.7/site-packages/srsly/_json_api.py", line 50, in read_json 

    file_path = force_path(location) 

  File "/usr/local/lib/python3.7/site-packages/srsly/util.py", line 21, in force_path 

    raise ValueError("Can't read file: {}".format(location)) 

ValueError: Can't read file: models/CLI_1/model0/accuracy.json 

我的培训和开发文档是使用 spacy.gold.docs_to_json() 生成的,使用以下函数保存为 json 文件:

def make_CLI_json(mock_docs, CLI_out_file_path): 
    CLI_json = docs_to_json(mock_docs) 
    with open(CLI_out_file_path, 'w') as json_file: 
        json.dump(CLI_json, json_file) 

我在http://www.jsonlint.com验证了它们都是有效的 json 。

我使用该函数创建了这些 json 源自的文档:

def import_from_doccano(jx_in_file_path, view=True): 
    annotations = load_jsonl(jx_in_file_path)     
    mock_nlp = English()     
    sentencizer = mock_nlp.create_pipe("sentencizer")     
    unlabeled = 0     
    DATA = []     
    mock_docs = [] 

    for anno in annotations:                      
        # get DATA (as used in spacy inline training)     
        if "label" in anno.keys():     
            ents = [tuple([label[0], label[1], label[2]])     
                               for label in anno["labels"]] 
        else: 
            ents = [] 

        DATUM = (anno["text"], {"entities": ents}) 
        DATA.append(DATUM) 

        # mock a doc for viz in displacy 
        mock_doc = mock_nlp(anno["text"]) 
        if "labels" in anno.keys():     
            entities = anno["labels"]     
            if not entities:     
                unlabeled += 1     
            ents = [(e[0], e[1], e[2]) for e in entities]     
            spans = [mock_doc.char_span(s, e, label=L) for s, e, L in ents]     
            mock_doc.ents = _cleanup_spans(spans)     
            sentencizer(mock_doc) 

            if view:     
                displacy.render(mock_doc, style='ent') 

        mock_docs.append(mock_doc)                      
    print(f'Unlabeled: {unlabeled}')                      
    return DATA, mock_docs 

我编写了上面的函数以在线训练所需的格式返回示例(例如,如https://github.com/explosion/spaCy/blob/master/examples/training/train_ner.py所示)以及形成这些“模拟”文档,以便我可以使用 displacy 和/或 CLI。对于后一个目的,我遵循https://github.com/explosion/spaCy/blob/master/spacy/cli/converters/jsonl2json.py中显示的代码有几个显着的差异。_cleanup_spans() 函数与示例中的函数相同。我没有使用 minibatch(),而是为我的每个标记注释制作了一个单独的文档。另外,(也许很重要?)我发现使用句子化器破坏了我的许多注释,可能是因为跨度以 _cleanup_spans() 函数无法正确修复的方式发生变化。删除语句器会导致 docs_to_json() 函数抛出错误。因此,在我的函数中(与链接示例不同),在将实体写入 每个文档后,我在每个文档上运行语句器,这样可以正确保留我的注释并允许 docs_to_json() 函数运行而不会产生任何抱怨。

在 import_from_doccano() 中调用的函数 load_jsonl 定义为:

def load_jsonl(input_path):     
    data = []     
    with open(input_path, 'r', encoding='utf-8') as f:     
        for line in f:     
            data.append(json.loads(line.replace('\n|\r',''), strict=False))     
    print('Loaded {} records from {}'.format(len(data), input_path))     
    print()     
    return data 

我的注释每个长度约为 10000 个字符或更少。它们是从 doccano 导出的

( https://doccano.herokuapp.com/ ) 作为 JSONL 使用以下格式:

{"id": 1, "text": "EU rejects ...", "labels": [[0,2,"ORG"], [11,17, "MISC"], [34,41,"ORG"]]}
{"id": 2, "text": "Peter Blackburn", "labels": [[0, 15, "PERSON"]]}
{"id": 3, "text": "President Obama", "labels": [[10, 15, "PERSON"]]}
...

使用以下函数将数据分为训练集和测试集:

def test_train_split(DATA, mock_docs, n_train):
    L = list(zip(DATA, mock_docs))
    random.shuffle(L)
    DATA, mock_docs = zip(*L)
    DATA = [i for i in DATA]
    mock_docs = [i for i in mock_docs]
    TRAIN_DATA = DATA[:n_train]
    train_docs = mock_docs[:n_train]
    TEST_DATA = DATA[n_train:] 
    test_docs = mock_docs[n_train:]
    return TRAIN_DATA, TEST_DATA, train_docs, test_docs

最后,每个都使用以下函数写入 json:

def make_CLI_json(mock_docs, CLI_out_file_path):
    CLI_json = docs_to_json(mock_docs)
    with open(CLI_out_file_path, 'w') as json_file:
        json.dump(CLI_json, json_file)

我不明白为什么调试显示 0 个培训文档和 0 个开发文档,或者为什么 train 命令失败。据我所知,JSON 看起来是正确的。我的数据格式不正确,还是发生了其他事情?任何帮助或见解将不胜感激。

这是我关于 SE 的第一个问题——如果我未能遵循某些或其他指导方针,请提前道歉。涉及到很多组件,所以我不确定如何生成一个最小的代码示例来复制我的问题。

环境

Mac OS 10.15 Catalina 一切都是 pip3 安装到用户路径 没有虚拟环境

| => python3 -m spacy info --markdown

## Info about spaCy

* **spaCy version:** 2.2.1
* **Platform:** Darwin-19.0.0-x86_64-i386-64bit
* **Python version:** 3.7.4
4

1 回答 1

3

这是 API 令人困惑的一个方面。出于内部/历史原因,spacy.gold.docs_to_json()生成一个仍需要包含在列表中的 dict 才能获得最终的训练格式。尝试:

srsly.write_json(filename, [spacy.gold.docs_to_json(docs)])

spacy debug-data还没有适当的模式检查,所以这比它应该的更令人沮丧/困惑。

于 2019-10-23T09:24:00.993 回答