how to increase number of mappers and reducers in mapreduce program in hadoop?

Question

I have a cluster of 3 nodes and I want to increase the number of mapper and reducer so that each node has 5 mapper and reducers. I used the code below but it did not work for me. Any help?

mapred-site.xml

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>vhost2456:9001</value>
<description>The host and port that the MapReduce job tracker runs
at.</description>
 </property>
 <property>
 <name>mapred.tasktracker.map.tasks.maximum</name>
 <value>15</value>
 </property>
 <property>
 <name>mapred.tasktracker.reduce.tasks.maximum</name>
 <value>15</value>
 </property>
 <property>
 <name>mapred.map.tasks</name>
 <value>5</value>
 </property>
 <property>
 <name>reduce.map.tasks</name>
 <value>5</value>
 </property>
 </configuration>

Answer 1

Looks like you have a mixture of configuration settings here:

These two define the number of map and reduce slots available on each slave node (running a task tracker):

<property>
  <name>mapred.tasktracker.map.tasks.maximum</name>
  <value>15</value>
</property>
<property>
  <name>mapred.tasktracker.reduce.tasks.maximum</name>
  <value>15</value>
</property>

Where as mapred.map.tasks is a (pretty much ignored) hint for a particular job as to how many map and reduce tasks to schedule for your job.

The final config property is malformed, i think you mean mapred.reduce.tasks which does control the number of reducers that will run for a particular job.

So currently it looks like you have 15 map and 15 reduce slots configured for a given task tracker (these values are for each task tracker, not for the entire cluster) - amend these values to 5. You'll also need to deploy this configuration change to all 3 of your cluster nodes, and finally you will need to restart the task trackers on all three nodes (for the change to take effect). You should be able to see the change in the Job tracker Web UI, under the number of map and reduce slots.