How to run Hadoop on OS X?

Question

I have a macbook Pro. I think I successfully installed hadoop-streaming but I have no clue where it is installed, and where the hdfs is. I have installed the single node version. I am using python to write code. I placed the mapper.py and reducer.py on my desktop and i have placed a big text file on the desktop as well, on which i will be running the programs. I am following this tutorial http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/ problem is I have no clue as to what path to type in section "Copy local example data to HDFS" in the tutorial. I successfully ran the python file individually and I have used the same code as the tutorial. I installed Hadoop using brew.

Thank you

Answer 1

which hadoop

or

echo $HADOOP_HOME 

will get you the path of installed hadoop (if you have followed all the installation steps)

you should be typing following command from directory at which your file to transferred is located. or you can define the entire path…

hadoop fs -copyFromLocal /local/path/file.txt /hdfs/path/file.txt

hadoop fs -put localfile1 localfile2 /user/hadoop/hadoopdir

more examples available @ [Hadoop Shell Commands]:https://hadoop.apache.org/docs/r0.18.3/hdfs_shell.html#put