How to open HDFS output file using gedit?

Question

I have installed and executed an mapreduce program successfully in my system(Ubuntu 14.04).

I can see the output file as,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -ls /user/hadoopuser/MapReduceSample-output
Found 3 items
-rw-r--r--   1 hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_SUCCESS
drwxr-xr-x   - hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_logs
-rw-r--r--   1 hadoopuser supergroup     880838 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/part-00000

And I can open it on terminal using following command,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -cat /user/hadoopuser/MapReduceSample-output/part-00000

I can see the output file on terminal, but I can't see the full result because my output has large amount of lines.

So I want to open it on gedit or nano.

Need Solution.

Answer 1

You can't access HDFS file from local machine(system user), so that you can't open HDFS file using gedit.

To open in gedit you have to copy to local machine.

To do that, open terminal(Ctrl+Alt+T) and use copyToLocal a Hadoop Shell Command to copy the output file into local machine.

Do the following,

hadoopuser@arul-PC:/usr/local/hadoop$ sudo bin/hadoop dfs -copyToLocal /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/

Now you can open the output file using gedit as follows,

$ sudo gedit /home/arul/Downloads/part-00000

Note :

1. My HDFS username is hadoopuser.
2. You can move a file from HDFS to local machine. The Hadoop Shell Command fs -mv allow to move different HDFS location.
3. For more Hadoop Shell Commands(click here).

Update (An another option to do the same from Y-Prithvi's post)

you can also use getmerge to copy HDFS file to local system.

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save

Answer 2

you can also use getmerge to copy HDFS file to local system.

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save

Answer 3

instead of looking for plugin. You can add jar files from $HADOOP_INSTALL/bin in eclipse and compiler issues must be gone.

Answer 4

Eclipse Setup for Hadoop Development

this should help