这里有关于如何训练模型的信息:
https://stanfordnlp.github.io/CoreNLP/depparse.html
示例命令:
java -Xmx12g edu.stanford.nlp.parser.nndep.DependencyParser -trainFile fr-ud-train.conllu -devFile fr-ud-dev.conllu -model new-french-UD-model.txt.gz -embedFile wiki.fr.vec -embeddingSize 300 -tlp edu.stanford.nlp.trees.international.french.FrenchTreebankLanguagePack -cPOS
您还需要训练一个词性模型:
https://nlp.stanford.edu/software/pos-tagger-faq.html
https://nlp.stanford.edu/nlp/javadoc/javanlp/edu/stanford/nlp/tagger/maxent/MaxentTagger.html
示例命令:
java -mx1g edu.stanford.nlp.tagger.maxent.MaxentTagger -props myPropertiesFile.props
您可以在文档中找到合适的培训文件样式。
示例文件:
## tagger training invoked at Sun Sep 23 19:24:37 PST 2018 with arguments:
model = english-left3words-distsim.tagger
arch = left3words,naacl2003unknowns,wordshapes(-1,1),distsim(/u/nlp/data/pos_tags_are_useless/egw4-reut.512.clusters,-1,1),distsimconjunction(/u/nlp/data/pos_tags_are_useless/egw4-reut.512.clusters,-1,1)
wordFunction = edu.stanford.nlp.process.AmericanizeFunction
trainFile = /path/to/training-data
closedClassTags =
closedClassTagThreshold = 40
curWordMinFeatureThresh = 2
debug = false
debugPrefix =
tagSeparator = _
encoding = UTF-8
iterations = 100
lang = english
learnClosedClassTags = false
minFeatureThresh = 2
openClassTags =
rareWordMinFeatureThresh = 10
rareWordThresh = 5
search = owlqn
sgml = false
sigmaSquared = 0.0
regL1 = 0.75
tagInside =
tokenize = true
tokenizerFactory =
tokenizerOptions =
verbose = false
verboseResults = true
veryCommonWordThresh = 250
xmlInput =
outputFile =
outputFormat = slashTags
outputFormatOptions =
nthreads = 1
这里有一个详尽的示例训练属性文件列表:
https://github.com/stanfordnlp/CoreNLP/tree/master/scripts/pos-tagger
如果您使用 Java 管道,则需要编写标记器或提供预标记的文本。
您可能对我们的 Python 项目感兴趣,该项目具有用于标记化、句子分割、词形还原和依赖解析的波兰模型。您也可以训练自己的模型:
https://github.com/stanfordnlp/stanfordnlp