Question

I have a simple Map/Reduce task. Mapper is called and executed normally but reducer is never called.

Configuration:

conf.setJobName("Index Builder");
conf.setSpeculativeExecution(false);

FileInputFormat.setInputPaths(conf, new Path(args[0].toString()));
FileOutputFormat.setOutputPath(conf, new Path(args[1].toString()));

conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(NullOutputFormat.class);

conf.setMapperClass(IndexMapper.class);
conf.setReducerClass(IndexReducer.class);

conf.setMapOutputKeyClass(NullWritable.class);
conf.setMapOutputValueClass(NullWritable.class);
conf.setOutputValueClass(NullWritable.class);
conf.setOutputKeyClass(NullWritable.class);

Mapper Signature:

public class IndexMapper extends MapReduceBase implements
  Mapper<LongWritable, Text, NullWritable, NullWritable> {

   @Override
   public void map(LongWritable key, Text val,
     OutputCollector<NullWritable, NullWritable> output,
     Reporter reporter) throws IOException {
        // MAP FUNCTION
   }

}

ٌReducer Signature:

public class IndexReducer extends MapReduceBase implements
  Reducer<NullWritable, NullWritable, NullWritable, NullWritable> {

   @Override
   public void reduce(NullWritable arg0, Iterator<NullWritable> arg1,
     OutputCollector<NullWritable, NullWritable> arg2, Reporter reporter)
     throws IOException {
      // REDUCE CODE
   }
}
Was it helpful?

Solution

You set output format as NullOutputFormat for the mapper, so it doesn't produce anything. So the reducer is never called as it needs some data to run.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top