java - MapReduce program not sharing string variables between main and mapper

Question

I am writing a mapreduce program in which the String created in Main method has to be shared in Mapper class. This is using New mapreduce api. I coded properly and set the variable using configuration in main method as below.

Configuration conf = new Configuration();
Job job = new Job(conf);

SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmsss");
String date = sdf.format(new Date());
String ImagesDir    = "/user/srini/images/"+ date;
conf.set("ImagesDir", ImagesDir);

and then I am picking the variable in Mapper class setup method as below. First created a variable in class as String OutputPath and then did the following in the setup.

Configuration conf = context.getConfiguration();
OutputPath  = conf.get("ImagesDir");

and used this variable in map method. The problem is, the value in variable OutputPath is null all the time. I have tried this using JobConf in Old mapred API long back and it worked fine. Some how, its going wrong here. What could have been wrong. Please help me..

Answer 1

不推荐使用Job构造函数，使用该Job.getInstance方法创建 Job 对象。

根据该getInstance方法的文档，它实际上复制了传递的Configuration对象，因此您在创建作业后对配置所做的任何修改对系统的任何部分都是不可见的。只需将配置设置移动到作业创建之前，如下所示：

Configuration conf = new Configuration();
conf.set("ImagesDir", ImagesDir);
Job job = Job.getInstance(conf);

Answer 2

您是否必须按照您的方式进行配置？如果没有，试试这个：

String ImagesDir = "/user/srini/images/"+ date;
FileOutputFormat.setOutputPath(job, new Path(ImageDir));

提醒一下，一般规则是将变量命名为image_diror imageDir。