0

我有一个使用 JobClient 提交作业的 mapreduce 代码。我不断收到这个空指针错误堆栈

12/12/10 12:42:44 INFO mapred.LocalJobRunner: OutputCommitter set in config null    
Exception in thread "main" java.lang.NullPointerException
        at org.apache.hadoop.mapred.JobClient$NetworkedJob.<init>(JobClient.java:226)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:924)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:818)
        at FTPIF.run(FTPIF.java:193)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at FTPIF.main(FTPIF.java:273)

让我遇到这个错误的代码是

JobClient j = new JobClient();
        j.init(conf);
        RunningJob check = j.submitJob(conf);

有任何想法吗 ？

4

2 回答 2

1

我也有这个问题。但是使用j.runJob(conf);效果很好。您也可以使用静态函数JobClient.runJob(conf);。不知道为什么，看了下JobClient的源码，找到了静态函数的实现runJob：

public static RunningJob runJob(JobConf job) throws IOException {
    JobClient jc = new JobClient(job);
    RunningJob rj = jc.submitJob(job);
    try {
        if (!jc.monitorAndPrintJob(job, rj)) {
            throw new IOException("Job failed!");
        }
    } catch (InterruptedException ie) {
        Thread.currentThread().interrupt();
    }
    return rj;
}

所以我submitJob以这种方式使用该功能：

JobClient jcli = new JobClient(jconf);
RunningJob rJob = jcli.submitJob(jconf);
while (true){
    Thread.sleep(5000);
    System.out.println(rJob);
    if (rJob.isComplete())
        break;
}

我工作！！也许这对你也有用。

于 2013-06-22T04:33:22.337 回答
0

我从来不明白为什么我会碰到空指针。设置 j.setConf(conf) 也不起作用。所以我runJob改用处理正在运行的作业。

RunningJob job= JobClient.runJob(conf);
于 2012-12-13T06:10:55.323 回答