java - 新手 hadoop .. 分布式缓存

Question

我有一个映射器

public class BuildGraph{
   public void config(JobConf job){ <==this block doesnt seems to be exexcuting at all :(
    super.configure(job);
    this.currentId = job.getInt("currentId",0);
    if (this.currentId!=0){ 
    // I call a method from differnt class to build a distributed cache
    }
   }
  public void map(....){
....  
}

}

现在是调用它的主要代码..

public void run( String params,curId){
 JobConf conf = new JobConf(classname.class);
 conf.setInt("currentId",299); <--note this i am setting the value here
 conf.setMapperClass(BuildGraph.class);
 //.... 
  JobClient.runJob(conf);
}

但问题是代码中的配置方法没有执行，好像“currentId”在主循环中返回 299，但在映射器类中根本没有设置。我究竟做错了什么

链接到完整代码 http://pastebin.com/DRPXUm62

Answer 1

看起来您没有使用正确的合同，因为您没有扩展MapReduceBase也没有实施Mapper. 该方法也应该被调用configure而不是config。尝试这样的事情：

public class BuildGraph extends MapReduceBase implements Mapper<K, V, K, V> {
    public void configure(JobConf job) {
        // this will get called once at the beginning of the task
    }

    public void map(...) {
        ...
    }
}