hadoop 1.1.2 - streaming jar not found

Question

I am getting the below error: when I try to execute mapreduce job written in Python.. I am not able to locate the streaming*.jar... Please suggest how this issue can be solved.. Can you please also guide me in writing bash file for the below commands..

hduser@hadoop-PC:~/hadoop/contrib$ hadoop jar contrib/streaming/hadoop-*streaming*.jar -file /home/hduser/mapper.py    -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py   -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output
Warning: $HADOOP_HOME is deprecated.

Exception in thread "main" java.io.IOException: Error opening job jar: contrib/streaming/hadoop-*streaming*.jar
    at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.io.FileNotFoundException: contrib/streaming/hadoop-*streaming*.jar (No such file or directory)
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:215)
    at java.util.zip.ZipFile.<init>(ZipFile.java:145)
    at java.util.jar.JarFile.<init>(JarFile.java:153)
    at java.util.jar.JarFile.<init>(JarFile.java:90)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

Answer 1

Well, I think, not hadoop-streaming.jar, it should be hadoop-streaming-1.1.2.jar, so try this:

hduser@hadoop-PC:~/hadoop$./bin/hadoop jar contrib/streaming/hadoop-streaming-1.1.2.jar** -file /home/hduser/mapper.py    -mapper /home/hduser/mapper.py -file /home/hduser/reducer.py   -reducer /home/hduser/reducer.py -input /user/hduser/gutenberg/* -output /user/hduser/gutenberg-output

Answer 2

Um, dumb question but did you verify that A) the file exists and B) you have permissions to access it? Also, try accessing the file like ./contrib/streaming/hadoop-streaming.jar or even the fully qualified path to it.

I just noticed that you were running your code from ~/hadoop/contrib. You need to take off the leading contrib from you paths