0

我制作了一个 CRF 模型。我的数据集有 24 个类,此时我刚刚开始,所以我的训练数据只有 1200 个标记/语料库。我已经训练了模型。在我的训练数据中,我使用了复数形式的标记,例如地址、照片、州、国家等。

现在在测试时,如果我给这个模型以句子形式的复数标记,那么它工作得很好,但是如果我以单数形式输入我的句子,比如照片、状态等,那么它不会给它分配任何标签。

crf 的这种行为看起来很奇怪。我已经探索了NER 特征工厂并使用了一些引理特征,但它也没有用。分享我austen.prop的模型形成。

# location of the training file
trainFile = training_data_for_ner.txt
# location where you would like to save (serialize) your
# classifier; adding .gz at the end automatically gzips the file,
# making it smaller, and faster to load
serializeTo = ner-model.ser.gz

# structure of your training file; this tells the classifier that
# the word is in column 0 and the correct answer is in column 1
map = word=0,answer=1,pos=2,lemma=3

# This specifies the order of the CRF: order 1 means that features
# apply at most to a class pair of previous class and current class
# or current class and next class.
maxLeft=1

# these are the features we'd like to train with
# some are discussed below, the rest can be
# understood by looking at NERFeatureFactory
useClassFeature=true
useWord=true
# word character ngrams will be included up to length 6 as prefixes
# and suffixes only 
useNGrams=true
noMidNGrams=true
maxNGramLeng=6
usePrev=true
useNext=true
useDisjunctive=true
useSequences=true
usePrevSequences=true
# the last 4 properties deal with word shape features
useTypeSeqs=true
useTypeSeqs2=true
useTypeySequences=true
wordShape=chris2useLC
# newly added features.
useLemmas=true
usePrevNextLemmas=true
useLemmaAsWord=true
useTags=true

最后四个功能是通过阅读添加的NER Feature Factory。如果有人可以帮助我解决这个问题,那么我将不胜感激。

4

1 回答 1

0

您应该使用词干标记重新训练它。参见https://github.com/stanfordnlp/CoreNLP/blob/master/src/edu/stanford/nlp/process/Stemmer.javamain方法)例如。

于 2016-12-29T13:24:50.637 回答