0

我已经运行了 map/reduce 作业，但它不起作用。我该如何解决这个错误？？你能告诉PLZ吗？

13/09/16 15:58:47 INFO mapred.JobClient: Task Id : attempt_201307081931_0006_m_000000_2, Status : FAILED
java.lang.ClassCastException: class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
at java.lang.Class.asSubclass(Class.java:3116)
at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:774)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:959)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:756)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)

我的源代码：

WordCountMapper.java

public class WordCountMapper extends Mapper<LongWritable, Text, Text, IntWritable> {

private final static IntWritable one = new IntWritable();
private Text word = new Text();

public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException{

    StringTokenizer itr = new StringTokenizer(value.toString());
    while(itr.hasMoreTokens()) {
        word.set(itr.nextToken());
        context.write(word, one);
    }
}
}

WordCountReducer.java

public class WordCountReducer extends Reducer<Text, IntWritable, Text, IntWritable> {

private IntWritable result = new IntWritable();

public void reduce(Text key, Iterable<IntWritable> values, Context context) throws IOException, InterruptedException{

    int sum = 0;
    for(IntWritable val : values){
        sum += val.get();
    }
    result.set(sum);
    context.write(key, result);
}
}

WordCount.java

public class WordCount {

public static void main(String[] args) throws Exception {

    Configuration conf = new Configuration();
    if(args.length != 2){
        System.err.println("Usage: WordCount <input> <output>");
        System.exit(2);
    }

    Job job = new Job(conf, "wordCount");

    job.setJarByClass(WordCount.class);
    job.setMapperClass(WordCountMapper.class);
    job.setReducerClass(WordCountReducer.class);

    job.setInputFormatClass(TextInputFormat.class);
    job.setOutputFormatClass(TextOutputFormat.class);

    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);

    FileInputFormat.addInputPath(job, new Path(args[0]));
    FileOutputFormat.setOutputPath(job, new Path(args[1]));

    job.waitForCompletion(true);
}
}
4

3 回答 3

15

我遇到了同样的异常，浪费了一天。原因是我在 main 方法上错误的导入声明。“import com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text”应该是“import org.apache.hadoop.io.Text”。

于 2013-10-05T07:26:46.347 回答
4

我认为，你错了“文本”类型的包依赖错误，应该是

`"import org.apache.hadoop.io.Text"
于 2013-09-23T12:45:06.767 回答
1

是的，我在 IDE 中使用 LocalRunner 测试 hadoop 作业时也遇到了这个问题。

我已经使用 CTRL + O（没有通知）而不是 org.apache.hadoop.io.Text 导入了“com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider.Text”。

于 2015-05-16T01:46:37.057 回答