Question

I have a simple Map/Reduce task. Mapper is called and executed normally but reducer is never called.

Configuration:

conf.setJobName("Index Builder");
conf.setSpeculativeExecution(false);

FileInputFormat.setInputPaths(conf, new Path(args[0].toString()));
FileOutputFormat.setOutputPath(conf, new Path(args[1].toString()));

conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(NullOutputFormat.class);

conf.setMapperClass(IndexMapper.class);
conf.setReducerClass(IndexReducer.class);

conf.setMapOutputKeyClass(NullWritable.class);
conf.setMapOutputValueClass(NullWritable.class);
conf.setOutputValueClass(NullWritable.class);
conf.setOutputKeyClass(NullWritable.class);

Mapper Signature:

public class IndexMapper extends MapReduceBase implements
  Mapper<LongWritable, Text, NullWritable, NullWritable> {

   @Override
   public void map(LongWritable key, Text val,
     OutputCollector<NullWritable, NullWritable> output,
     Reporter reporter) throws IOException {
        // MAP FUNCTION
   }

}

ٌReducer Signature:

public class IndexReducer extends MapReduceBase implements
  Reducer<NullWritable, NullWritable, NullWritable, NullWritable> {

   @Override
   public void reduce(NullWritable arg0, Iterator<NullWritable> arg1,
     OutputCollector<NullWritable, NullWritable> arg2, Reporter reporter)
     throws IOException {
      // REDUCE CODE
   }
}
Était-ce utile?

La solution

You set output format as NullOutputFormat for the mapper, so it doesn't produce anything. So the reducer is never called as it needs some data to run.

Licencié sous: CC-BY-SA avec attribution
Non affilié à StackOverflow
scroll top