1

我正在尝试运行 LDA 从 txt 文件中生成一些主题，如下所示：

文档1 标签1 森林=3.4 树=5 木=2.85 锤子=1 颜色=1 叶=1.5

文档2 标签2 森林=10 树=5 木=2.75 锤子=1 颜色=4 叶=1

文档3 标签3 森林=19 树=0.90 木=2 锤子=2 颜色=9 叶=4.3

文档4 标签4 森林=4 树=5 木=10 锤子=1 颜色=6 叶=3

文件中的每个数值都是每个特征（例如，森林、树）出现次数乘以给定惩罚的指示。

要从这样的文件生成实例，我使用以下 Java 代码：

String lineRegex = "^(\\S*)[\\s,]*(\\S*)[\\s,]*(.*)$";

String dataRegex = "[\\p{L}([0-9]*\\.[0-9]+|[0-9]+)_\\=]+";

InstanceList generateInstances(String dataPath) throws UnsupportedEncodingException, FileNotFoundException {
   	 
ArrayList<Pipe> pipeList = new ArrayList<Pipe>();
       
       pipeList.add(new Target2Label());
       pipeList.add( new CharSequenceLowercase() ); 
       pipeList.add( new Input2CharSequence() ); 
       pipeList.add( new CharSequence2TokenSequence(Pattern.compile(dataRegex)) );
       /*pipeList.add( new TokenSequenceRemoveStopwords(new File(stopwordListPath), "UTF-8", 
       		      false, false, false) );*/
       pipeList.add( new TokenSequenceParseFeatureString(true,true,"=") );
       pipeList.add( new PrintInputAndTarget());

       InstanceList instances = new InstanceList (new SerialPipes(pipeList));

       Reader fileReader = new InputStreamReader(new FileInputStream(new File(dataPath)), 
     		                                    "UTF-8");
       
       instances.addThruPipe(new CsvIterator (fileReader, Pattern.compile(lineRegex),
                                              3, 2, 1)); 
      
       return instances;
   }

然后，我使用指令 model.addInstances(generatedInstances) 将如此生成的实例添加到我的模型中。生成的输出如下所述。它包含由指令 model.addInstances(generatedInstances) 引起的错误。调试我的代码显示与模型关联的字母表为空。我使用了错误的迭代器吗？谁能帮我修复我的代码？

name: document1
target: label1
input: TokenSequence [forest=3.4 feature(forest)=3.4  span[0..10], tree=5 feature(tree)=5.0  span[11..17], wood=2.85 feature(wood)=2.85  span[18..27], hammer=1 feature(hammer)=1.0  span[28..36], colour=1 feature(colour)=1.0  span[37..45], leaf=1.5 feature(leaf)=1.5  span[46..54]]
Token#0:forest=3.4 feature(forest)=3.4  span[0..10]
Token#1:tree=5 feature(tree)=5.0  span[11..17]
Token#2:wood=2.85 feature(wood)=2.85  span[18..27]
Token#3:hammer=1 feature(hammer)=1.0  span[28..36]
Token#4:colour=1 feature(colour)=1.0  span[37..45]
Token#5:leaf=1.5 feature(leaf)=1.5  span[46..54]

name: document2
target: label2
input: TokenSequence [forest=10 feature(forest)=10.0  span[0..9], tree=5 feature(tree)=5.0  span[10..16], wood=2.75 feature(wood)=2.75  span[17..26], hammer=1 feature(hammer)=1.0  span[27..35], colour=4 feature(colour)=4.0  span[36..44], leaf=1 feature(leaf)=1.0  span[45..51]]
Token#0:forest=10 feature(forest)=10.0  span[0..9]
Token#1:tree=5 feature(tree)=5.0  span[10..16]
Token#2:wood=2.75 feature(wood)=2.75  span[17..26]
Token#3:hammer=1 feature(hammer)=1.0  span[27..35]
Token#4:colour=4 feature(colour)=4.0  span[36..44]
Token#5:leaf=1 feature(leaf)=1.0  span[45..51]

name: document3
target: label3
input: TokenSequence [forest=19 feature(forest)=19.0  span[0..9], tree=0.90 feature(tree)=0.9  span[10..19], wood=2 feature(wood)=2.0  span[20..26], hammer=2 feature(hammer)=2.0  span[27..35], colour=9 feature(colour)=9.0  span[36..44], leaf=4.3 feature(leaf)=4.3  span[45..53]]
Token#0:forest=19 feature(forest)=19.0  span[0..9]
Token#1:tree=0.90 feature(tree)=0.9  span[10..19]
Token#2:wood=2 feature(wood)=2.0  span[20..26]
Token#3:hammer=2 feature(hammer)=2.0  span[27..35]
Token#4:colour=9 feature(colour)=9.0  span[36..44]
Token#5:leaf=4.3 feature(leaf)=4.3  span[45..53]

name: document4
target: label4
input: TokenSequence [forest=4 feature(forest)=4.0  span[0..8], tree=5 feature(tree)=5.0  span[9..15], wood=10 feature(wood)=10.0  span[16..23], hammer=1 feature(hammer)=1.0  span[24..32], colour=6 feature(colour)=6.0  span[33..41], leaf=3 feature(leaf)=3.0  span[42..48]]
Token#0:forest=4 feature(forest)=4.0  span[0..8]
Token#1:tree=5 feature(tree)=5.0  span[9..15]
Token#2:wood=10 feature(wood)=10.0  span[16..23]
Token#3:hammer=1 feature(hammer)=1.0  span[24..32]
Token#4:colour=6 feature(colour)=6.0  span[33..41]
Token#5:leaf=3 feature(leaf)=3.0  span[42..48]

Coded LDA: 5 topics, 3 topic bits, 111 topic mask
Exception in thread "main" java.lang.NullPointerException
at cc.mallet.topics.ParallelTopicModel.addInstances(ParallelTopicModel.java:217)
at mallet.examples.TopicModel3.runLDA(MyTopicModel.java:106)
at mallet.examples.TopicModel3.main(MyTopicModel.java:57)

提前致谢。

4

0 回答 0