Error starting hadoop 2.2.0 minicluster: java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/MiniYARNCluster

Question

I'm using Apache Hadoop 2.2.0 on MacOS development environment. When trying to run hadoop minicluster as described in apache docs:

hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar  minicluster

I'm getting the error

java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/server/MiniYARNCluster at org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:170) at org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:129) at . . . . . . .

Any ideas how to solve this?

Answer 1

It looks like I found the solution: The jar file containing MiniYARNCluster is not part of Hadoop classpath by default, it must be added explicitly into HADOOP_CLASSPATH. The snippet below worked fine:

export HADOOP_CLASSPATH=./share/hadoop/yarn/test/hadoop-yarn-server-tests-2.2.0-tests.jar
./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar  minicluster

Answer 2

I know this is an old post, but I recently needed to solve this problem and couldn't find a complete solution out there for MR2 and YARN testing, so I wanted to share what I found

http://www.lopakalogic.com/articles/hadoop-articles/hadoop-testing-with-minicluster/

Hope it helps someone