您可能想考虑使用 FACTORIE 工具包 ( http://github.com/factorie/factorie )。它是一个用于机器学习和图形模型的通用库,恰好包含一套广泛的自然语言处理组件(标记化、标记规范化、形态分析、句子分割、词性标记、命名实体识别、依赖解析、提及发现,共指)。
此外,它完全用 Scala 编写,并在 Apache 许可证下发布。
文档目前很少,但会在未来几个月内得到改进。
例如,基于 Maven 的安装完成后,您可以在命令行中键入:
bin/fac nlp --pos1 --parser1 --ner1
启动一个套接字侦听的多线程 NLP 服务器。然后通过将纯文本传输到其套接字号来查询它:
echo "Mr. Jones took a job at Google in New York. He and his Australian wife moved from New South Wales on 4/1/12." | nc localhost 3228
那么输出是
1 1 Mr. NNP 2 nn O
2 2 Jones NNP 3 nsubj U-PER
3 3 took VBD 0 root O
4 4 a DT 5 det O
5 5 job NN 3 dobj O
6 6 at IN 3 prep O
7 7 Google NNP 6 pobj U-ORG
8 8 in IN 7 prep O
9 9 New NNP 10 nn B-LOC
10 10 York NNP 8 pobj L-LOC
11 11 . . 3 punct O
12 1 He PRP 6 nsubj O
13 2 and CC 1 cc O
14 3 his PRP$ 5 poss O
15 4 Australian JJ 5 amod U-MISC
16 5 wife NN 6 nsubj O
17 6 moved VBD 0 root O
18 7 from IN 6 prep O
19 8 New NNP 9 nn B-LOC
20 9 South NNP 10 nn I-LOC
21 10 Wales NNP 7 pobj L-LOC
22 11 on IN 6 prep O
23 12 4/1/12 NNP 11 pobj O
24 13 . . 6 punct O
当然,所有这些功能也有一个编程 API。
import cc.factorie._
import cc.factorie.app.nlp._
val doc = new Document("Education is the most powerful weapon which you can use to change the world.")
DocumentAnnotatorPipeline(pos.POS1).process(doc)
for (token <- doc.tokens)
println("%-10s %-5s".format(token.string, token.posLabel.categoryValue))
将输出:
Education NN
is VBZ
the DT
most RBS
powerful JJ
weapon NN
which WDT
you PRP
can MD
use VB
to TO
change VB
the DT
world NN
. .