map reduce word count example doesn't work
-
12-11-2019 - |
Question
I try to implement the word count example by myself, here's my implementation of the mapper:
public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> {
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
Text word = new Text();
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()) {
word.set(tokenizer.nextToken());
context.write(word, new IntWritable(1));
}
}
}
and reducer:
public static class Reduce extends Reducer<Text, IntWritable, Text, IntWritable> {
public void reduce(Text key, Iterator<IntWritable> values, Context context) throws IOException, InterruptedException {
int sum = 0;
while (values.hasNext())
sum += values.next().get();
context.write(key, new IntWritable(sum));
}
}
but the output I get for executing this code looks like the output of mapper only, for example, if the input is "hello world hello", the output would be
hello 1
hello 1
world 1
I also use combiner between mapping and reducing. Can anyone explain me what's wrong with this code?
Thanks a lot!
No correct solution
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow