Can not set up Eclipse plugins, Hadoop Hortonworks 2.0

Question

I got Hortonworks Sandbox 2.0 and it is running fine. Now I want to try to connect from Eclipse, but I am unable to.

• Here is the Hadoop location on Eclipse:
  Map/Reduce master: host: localhost, port: 50020
  DFS Master: host: localhost, port:50040

The error is:

Call to localhost/127.0.0.1:50040 failed on connection exception: java.net.ConnectionException: Connection refused: no further information.

I think the error could be a mismatched Hadoop plugin since I am using Hadoop Eclipse plugin 1.2.0 but I could not find an Eclipse plugin for Hadoop 2.2

The answer from thread How to use java to connect hadoop 2.2.0 server says that currently there is no Eclipse plugin for Hadoop 2.2.0? Can someone guide me through this?

Answer 1

Jacky, there are different ports for different services.

50070 is HDFS service which every Hadoop system has (therefore I suggested it).

8888 is likely just a hortonworks' port for some specific web service.

50030 is Jobtracker port

First of all, make sure that you can connect/communicate with your VirtualBox (host os can access guest services). If so, find out what service/port you need e.g. Jobtracker API error - Call to localhost/127.0.0.1:50030 failed on local exception: java.io.EOFException

It can be a lot of work - so if you know Cloudera distribution has all things you need - go for Cloudera

Answer 2

Hadoop 2.2.0 version don't use jobtracker as a whole. The yarn has split the work of job tracker in two half see apache hadoop documentation.

First of all go to the mapred-site.xml add the below properties with the other mentioned in all installation steps:-

mapreduce.jobtracker.address

localhost:54311

mapreduce.jobtracker.http.address

0.0.0.0:50030

mapreduce.jobhistory.address

0.0.0.0:10020

mapreduce.jobhistory.webapp.address

0.0.0.0:19888

and after that configure your hadoop location as:-

Map/Reduce (V2) Master

Host: localhost

port: 54311

DFS Master

check the checkbox use M/R master host

port: 9000**

Now every thing will be fine.

Answer 3

I found myself in similar situation when I was unable to connect to Hive server in HortonWorks sandbox. What I found that the Virtual Image you use for SandBox uses NAT for networking. So what it means that the IP address of the Guest OS (Sandbox in this case) is same as the machine it runs on. To enable effective communication, there is Port Forwarding in visualization software. The ports that are default configured for Hadoop in Sandbox are mapped to different (or sometime same) ports by default. You can check the port forwarding and the configured rules to reach to specific service/port from the Host operating system. Now, about the eclipse plugin for Hadoop 2.2.0, I am yet to be successful to find how to do it. So I will post more as I go along my Hadoop development journey.