com.mongodb.hadoop.MongoOutputFormat not found when submit MapReduce job in Hadoop

Question

I follow this tutorial http://www.mongodb.org/display/DOCS/Hadoop+Quick+Start to build mongodb-hadoop. And i try to build Treasury Yield example ( My Hadoop version is 0.20.2. ) , but I got the following error when I submit the MapReduce job :

12/10/24 14:01:09 INFO util.MongoTool: Created a conf: 'Configuration: core-default.xml, core-site.xml, mongo-defaults.xml, mongo-treasury_yield.xml' on {class com.mongodb.hadoop.examples.treasury.TreasuryYieldXMLConfig} as job named '<unnamed job="" mongotool="">'
12/10/24 14:01:10 INFO util.MongoTool: Mapper Class: class com.mongodb.hadoop.examples.treasury.TreasuryYieldMapper
12/10/24 14:01:10 INFO util.MongoTool: Setting up and running MapReduce job in foreground, will wait for results. {Verbose? true}
12/10/24 14:01:10 INFO util.MongoSplitter: Calculate Splits Code ... Use Shards? false, Use Chunks? true; Collection Sharded? true
12/10/24 14:01:10 INFO util.MongoSplitter: Creation of Input Splits is enabled.
12/10/24 14:01:10 INFO util.MongoSplitter: Sharding mode calculation entering.
12/10/24 14:01:10 INFO mapred.JobClient: Running job: job_201208171702_0054
12/10/24 14:01:11 INFO mapred.JobClient: map 0% reduce 0%
12/10/24 14:01:17 INFO mapred.JobClient: Task Id : attempt_201208171702_0054_m_000116_0, Status : FAILED

java.lang.RuntimeException: java.lang.ClassNotFoundException: com.mongodb.hadoop.MongoOutputFormat 
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809) 
    at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:193) 
    at org.apache.hadoop.mapred.Task.initialize(Task.java:413) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:288) 
    at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.ClassNotFoundException: com.mongodb.hadoop.MongoOutputFormat 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:264) 
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762) 
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807) 
... 4 more

I cannot get the Treasury Yield example to run... Any help would be appreciated.

Answer 1

I got the same problem. Please make sure you have copied the required jar files(mongo-hadoop-core_cdh3u3-1.0.0.jar, mongo-2.10.1.jar) in hadoop lib folder. After that you will need to restart hadoop node for it to put it in its classpath. And then it should run.

It worked for me.