Question

I am trying to process a csv file. In map() I will be sending each column as a value to reducer. In reducer I will be finding the min and max and further processing .After processing i need to context .write the value.But trying to output in a specific manner.

In reducer I am processing each column but i need to output the entire row ie values of col1 ,values of col2, values of col3,values of col4.

Sample data after processing from reducer

in reducer
0.62
0.38
0.9
in reducer
0.24
0.59
0.1
in reducer
0.5
1.0
0.2
in reducer
0.0
0.0
0.3

Expected output

 0.62,0.24,0.5,0.5
 0.38,0.59,1.0,0.0
 0.9,0.1,0.2,0.3

Is it possible to handle each different key in reducer for each iteration

key1value1,key2value1,key3value1
key1value2,key2value2,key3value2

Any suggesions

Was it helpful?

Solution

Using MapReduce chaining you can Perform this

write reducer output in single line using Text type Like

0.62,0.38,0.9
0.24,0.59,0.1

and again perform mapreduce on output file of first MapReduce.

public void map(Text key, Text value,OutputCollector<Text, Text> output,
Reporter reporter) throws IOException {
  String temp=new String(value);
  String token[]=temp.split(",");
  output.collect(new Text("max"),new text(token[0]));
  output.collect(new Text("min"),new text(token[1]));
  output.collect(new Text("result"),new text(token[2]));
}

public void reduce(Text key, Iterator<Text> values,
    OutputCollector<Text, Text> output, Reporter reporter)
    throws IOException {
StringBuffer sb=new StringBuffer();
while (values.hasNext())
   sb.append(values.next().toString() + ",");
output.collect(new Text(""),new Text(sb.toString()))



}
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top