CDH WebHDFS request redirects to local address on EC2

Question

I am trying to setup an enviroment where I run some of my backend locally, and send requests to an EC2 instance from my local computer. I have CDH 4.5 setup, and it works OK. When I run the following request

curl --negotiate -i -L -u:hdfs http://ec2-xx-xx-xx-xx.eu-west-1.compute.amazonaws.com:50070/webhdfs/v1/tmp/test.txt?op=OPEN

This works from any EC2 instance in that region but does not work outside that. If I try locally it would return the following error

curl: (6) Could not resolve host: ip-xx-xx-xx-xx.eu-west-1.compute.internal

Not sure where I can set this not to redirect the call this way?

Many thanks

Answer 1

The easiest and fastest way to solve this problem is to configure your client hosts file to map the internal address to the external address.

WebHDFS uses the host name configured in hdfs-site.xml which is configured automatically by the Cloudera agent on that datanode. I don't know of a way to override the configured hostname for each datanode in CDH.