Question

I'm trying to set up a single node cluster using Cloudera Manager, here's the latest problem I'm having.

When I try to start all services, HDFS won't start, it appears to be a problem with the Datanode - the fatal error message it gives is

Initialization failed for block pool Block pool BP-1896453393-127.0.1.1-1382183730690 (storage id DS-185043233-127.0.1.1-50010-1382183775494) service to Hadoop1/127.0.1.1:8022
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException): Datanode denied communication with namenode: DatanodeRegistration(127.0.0.1, storageID=DS-185043233-127.0.1.1-50010-1382183775494, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=cluster25;nsid=1577346304;c=0)
at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:656)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3495)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:899)
at 
...

When I run the host inspector, it gives the following warning:

The following errors were found while checking /etc/hosts...

The hostname Hadoop1 maps to 127.0.1.1 in /etc/hosts. 
This setting, typically found in Debian and Ubuntu environments, typically causes issues. 
We recommend either using the standard loopback address (127.0.0.1) or specifying the appropriate IP address for your host.

From googling, it seems the problem is related to the IP address used, and the root of the problem might be in my /etc/hosts file, which looks like this.

127.0.0.1   localhost
127.0.1.1   Hadoop1

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters 
Was it helpful?

Solution

Got this sorted, instead of using 127.0.1.1 in the /etc/hosts file I used the IP address of the machine which is in the Point-to-Point section of ifconfig.

The output of ifconfig will have a section like

ppp0      Link encap:Point-to-Point Protocol  
          inet addr:xxx.xxx.xxx.xxx  P-t-P:xx.xx.xx.xx  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1440  Metric:1
          RX packets:800950 errors:1 dropped:0 overruns:0 frame:0
          TX packets:454479 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:1115444967 (1.1 GB)  TX bytes:32007110 (32.0 MB)

Take the inet addr: ip from this.

Or run

ifconfig | grep P-t-P

for the exact line.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top