You set output format as NullOutputFormat for the mapper, so it doesn't produce anything. So the reducer is never called as it needs some data to run.
Hadoop Mapreduce: Reducer never called
-
20-06-2023 - |
Question
I have a simple Map/Reduce task. Mapper is called and executed normally but reducer is never called.
Configuration:
conf.setJobName("Index Builder");
conf.setSpeculativeExecution(false);
FileInputFormat.setInputPaths(conf, new Path(args[0].toString()));
FileOutputFormat.setOutputPath(conf, new Path(args[1].toString()));
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(NullOutputFormat.class);
conf.setMapperClass(IndexMapper.class);
conf.setReducerClass(IndexReducer.class);
conf.setMapOutputKeyClass(NullWritable.class);
conf.setMapOutputValueClass(NullWritable.class);
conf.setOutputValueClass(NullWritable.class);
conf.setOutputKeyClass(NullWritable.class);
Mapper Signature:
public class IndexMapper extends MapReduceBase implements
Mapper<LongWritable, Text, NullWritable, NullWritable> {
@Override
public void map(LongWritable key, Text val,
OutputCollector<NullWritable, NullWritable> output,
Reporter reporter) throws IOException {
// MAP FUNCTION
}
}
ٌReducer Signature:
public class IndexReducer extends MapReduceBase implements
Reducer<NullWritable, NullWritable, NullWritable, NullWritable> {
@Override
public void reduce(NullWritable arg0, Iterator<NullWritable> arg1,
OutputCollector<NullWritable, NullWritable> arg2, Reporter reporter)
throws IOException {
// REDUCE CODE
}
}
Solution
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow