Class not found exception in eclipse wordcount program

Question

I am running a word count program from eclipse, it says class not found. I exported same program as jar file and executed from command line, it's working fine.

Here is the error stack trace

14/02/14 23:46:16 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
    14/02/14 23:46:16 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
    14/02/14 23:46:16 INFO input.FileInputFormat: Total input paths to process : 1
    14/02/14 23:46:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    14/02/14 23:46:16 WARN snappy.LoadSnappy: Snappy native library not loaded
    14/02/14 23:46:17 INFO mapred.JobClient: Running job: job_201402142205_0013
    14/02/14 23:46:18 INFO mapred.JobClient:  map 0% reduce 0%
    14/02/14 23:46:28 INFO mapred.JobClient: Task Id : attempt_201402142205_0013_m_000000_0, Status : FAILED
    java.lang.RuntimeException: java.lang.ClassNotFoundException: hadoop.wc.WordCount$TokenCounterMapper
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
        at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:718)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:394)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)
    Caused by: java.lang.ClassNotFoundException: hadoop.wc.WordCount$TokenCounterMapper
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:249)
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:855)
        ... 8 more

attempt_201402142205_0013_m_000000_0: 2014-02-14 23:46:20.815 java[8633:1003] Unable to load realm info from SCDynamicStore
14/02/14 23:46:28 INFO mapred.JobClient: Task Id : attempt_201402142205_0013_m_000001_0, Status : FAILED

Job configuration

  Job job = new Job(conf, "WordCount");
  job.setJarByClass(WordCount.class);
  job.setMapperClass(TokenCounterMapper.class);
  job.setReducerClass(TokenCounterReducer.class);
  job.setNumReduceTasks(2);
  job.setOutputKeyClass(Text.class);
  job.setOutputValueClass(IntWritable.class);

Thanks for your help.

Answer 1

If you are trying to run a job directly from eclipse then you need to first export your project as a runnable jar. You can place this jar at any location in your machine. You just need to specify the location of the jar in your main method of driver class as below:

conf.set("mapred.jar","D:\mapred_example.jar");

Hope this helps!