Forcing Hadoop to run the application with multiple reducers

Question

I have a Hadoop Map-reduce application with the following line in its code (which sets the number of reducers to 1).

job.setNumReduceTasks(1)

I would like to run this application with multiple reducers so I tried adding the following XML code to $HADOOP_CONF_DIR/mapred-site.xml

 <property>
  <name>mapreduce.job.reduces</name>
  <value>2</value>
 </property>

However, the code still runs with 1 reducer. Is there a way to force the hadoop application to run with 2 reducers or does the priority of the application code is always higher?

Answer 1

You'll need to amend the code in this case as the call to job.setNumReduceTasks(1) takes priority over any configuration or command line passed property

Answer 2

You can pass number of reducers when you submit the application (via command line). Passing configuration parameters via command line has the highest priority over passing it via application code (2nd highest priority) and via xml files (3rd priority)

Use -D option and set the values.

E.G: $ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount /usr/joe/wordcount/input /usr/joe/wordcount/output -D mapred.reduce.tasks=2