这是我在 map/reduce 中的第一个程序。我尝试计算文件中元音和辅音的数量,而不是传统的字数统计程序。下面是我的代码。
映射器:
公共类 VowelConsMapper 扩展 Mapper {
public void map(LongWritable mapKey,Text mapValue,Context context) throws IOException, InterruptedException{
String line = mapValue.toString();
String[] letters = line.split("");
for(String letter : letters){
System.out.println(letter);
if(letter!=" "){
if(isVowel(letter))
context.write(new Text("Vowel"), new IntWritable(1));
else
context.write(new Text("Consonant"), new IntWritable(1));
}
}
}
private boolean isVowel(String letter) {
// TODO Auto-generated method stub
if(letter.equalsIgnoreCase("a")||letter.equalsIgnoreCase("e")||letter.equalsIgnoreCase("i")||letter.equalsIgnoreCase("o")||letter.equalsIgnoreCase("u"))
return true;
else
return false;
}
}
减速器:
public class VowelConsReducer extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reducer(Text letterType,Iterable<IntWritable> values,Context context) throws IOException, InterruptedException{
int sum = 0;
for(IntWritable value:values){
sum += value.get();
}
System.out.println(letterType+" "+sum);
context.write(letterType, new IntWritable(sum));
}
}
司机:
公共类 VowelConsDriver {
public static void main(String[] args) throws IOException, InterruptedException, ClassNotFoundException {
Configuration conf = new Configuration();
conf.addResource(new Path("/home/hadoop/hadoop-1.0.3/conf/core-site.xml"));
Job job = new Job(conf,"VowelConsDriver");
job.setJarByClass(VowelConsDriver.class);
job.setMapperClass(VowelConsMapper.class);
job.setReducerClass(VowelConsReducer.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);
// TODO: specify output types
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path("/user/hadoop/WCinput"));
FileOutputFormat.setOutputPath(job, new Path("/user/hadoop/VowelConsOP1"));
job.waitForCompletion(true);
}
}
它给出以下 o/p:辅音 1 辅音 1 辅音 1 .................................... …………………………………………………………………………………… ..... 元音 1 元音 1 元音 1 元音 1 ...................................................... ………………………………………………………………………… ……
我期待每个类别的辅音和元音总数对不起,如果我没有正确格式化代码......提前谢谢!