If you just want to write a list of JSON objects to HDFS without caring about the notion of key/value, you could just use a NullWritable
in your Reducer
output value:
public static class TokenCounterReducer extends Reducer<Text, Text, Text, NullWritable> {
public void reduce(Text key, Iterable<Text> values, Context context)
throws IOException, InterruptedException {
for (Text value : values) {
JSONObject jsn = new JSONObject();
....
context.write(new Text(jsn.toString()), null);
}
}
}
Note that you will need to change your job configuration to do:
job.setOutputValueClass(NullWritable.class);
By writing your JSON object to HDFS I understood that you want to store a String representation of your JSON which I'm describing above. If you wanted to store a binary representation of your JSON into HDFS you would need to use a SequenceFile
. Obviously you could write your own Writable
for this but I feel it's just easier like this if you intend to have a simple String representation.