문제

I have a simple Map/Reduce task. Mapper is called and executed normally but reducer is never called.

Configuration:

conf.setJobName("Index Builder");
conf.setSpeculativeExecution(false);

FileInputFormat.setInputPaths(conf, new Path(args[0].toString()));
FileOutputFormat.setOutputPath(conf, new Path(args[1].toString()));

conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(NullOutputFormat.class);

conf.setMapperClass(IndexMapper.class);
conf.setReducerClass(IndexReducer.class);

conf.setMapOutputKeyClass(NullWritable.class);
conf.setMapOutputValueClass(NullWritable.class);
conf.setOutputValueClass(NullWritable.class);
conf.setOutputKeyClass(NullWritable.class);

Mapper Signature:

public class IndexMapper extends MapReduceBase implements
  Mapper<LongWritable, Text, NullWritable, NullWritable> {

   @Override
   public void map(LongWritable key, Text val,
     OutputCollector<NullWritable, NullWritable> output,
     Reporter reporter) throws IOException {
        // MAP FUNCTION
   }

}

ٌReducer Signature:

public class IndexReducer extends MapReduceBase implements
  Reducer<NullWritable, NullWritable, NullWritable, NullWritable> {

   @Override
   public void reduce(NullWritable arg0, Iterator<NullWritable> arg1,
     OutputCollector<NullWritable, NullWritable> arg2, Reporter reporter)
     throws IOException {
      // REDUCE CODE
   }
}
도움이 되었습니까?

해결책

You set output format as NullOutputFormat for the mapper, so it doesn't produce anything. So the reducer is never called as it needs some data to run.

라이센스 : CC-BY-SA ~와 함께 속성
제휴하지 않습니다 StackOverflow
scroll top