I am having a strange issue. I have my own implementation of filesystem instead of default distributedfilesystem. I have added my filesystem in fs.default.name and impl. When hadoop execution is started(teragen program),I could see that writes and reads are happening fine for files job.xml,job.jar etc from JobTracker.
But, once mapping tasks are allocated in jobtracker , it gives ioexception saying that split class is not found. I verified that all classes are present in the examples jar.
Also, to narrow down the issue, I retried the exactly same setup using default distributed system by just changing fs.default.name. It works perfectly!!.
Command
bin/hadoop jar hadoop-examples-1.2.1.jar teragen 100000 /user/hduser/terasort-input
Initially I felt this to be a classpath issue but, if same setup works for default distributed system, where can be the issue ?. How can my implementation in filesystem affect jobtracker and classpath?
I really appreciate your help.
2013-11-06 12:30:35,018 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201311061227_0001_m_000000_0: java.io.IOException: Split class PLLorg.apache.hadoop.examples.terasort.TeraGen$RangeInputFormat$RangeInputSplit not found
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:381)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:406)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:416)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: PLLorg.apache.hadoop.examples.terasort.TeraGen$RangeInputFormat$RangeInputSplit
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:266)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:810)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:379)