Failed to get system directory - hadoop

StackOverflow https://stackoverflow.com/questions/17916602

Question

Using hadoop multinode setup (1 mater , 1 salve)

After starting up start-mapred.sh on master , i found below error in TT logs (Slave an)

org.apache.hadoop.mapred.TaskTracker: Failed to get system directory

can some one help me to know what can be done to avoid this error

I am using Hadoop 1.2.0 jetty-6.1.26 java version "1.6.0_23"

mapred-site.xml file 

<configuration>
<property>
  <name>mapred.job.tracker</name>
  <value>master:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
</property>
<property>
  <name>mapred.map.tasks</name>
    <value>1</value>
      <description>
          define mapred.map tasks to be number of slave hosts
</description>
</property>

<property>
  <name>mapred.reduce.tasks</name>
    <value>1</value>
      <description>
          define mapred.reduce tasks to be number of slave hosts
</description>
</property>

</configuration>

core-site.xml

<configuration>
<property>
  <name>fs.default.name</name>
  <value>hdfs://master:54310</value>
  <description>The name of the default file system.  A URI whose
  scheme and authority determine the FileSystem implementation.  The
  uri's scheme determines the config property (fs.SCHEME.impl) naming
  the FileSystem implementation class.  The uri's authority is used to
  determine the host, port, etc. for a filesystem.</description>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/home/hduser/workspace</value>
</property>
</configuration>
Was it helpful?

Solution

It seems that you just added hadoop.tmp.dir and started the job. You need to restart the Hadoop daemons after adding any property to the configuration files. You have specified in your comment that you added this property at a later stage. This means that all the data and metadata along with other temporary files is still in the /tmp directory. Copy all those things from there into your /home/hduser/workspace directory, restart Hadoop and re run the job.

Do let me know the result. Thank you.

OTHER TIPS

If, it is your windows PC and you are using cygwin to run Hadoop. Then task tracker will not work.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top