Possible encrypted secure comminication to WebHDFS via HTTPS?

Question

I'm using Cloudera Hadoop environment.

I've been familiar with WebHDFS over the past few days. I'm using it to perform various HDFS operations through REST API.

But, now i need to secure the communication to WebHDFS i.e using SSL/HTTPS.

Is this possible ? If yes, are there any defined steps to achieve this ?

PS : Currently, I don't want to go for using Kerberos.

Answer 1

There are some properties you can set in hdfs-site.xml to enable HTTPS/SSL connections to the name node and data nodes:

• http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml
  • See the dfs.https.enable property
  • See the dfs.https.server.keystore.resource and dfs.client.https.keystore.resource properties for an ssl-server.xml (or client) file which defines the key and trustore paths / password etc (there are example ssl-server and ssl-client files in your conf folder)

You'll need to set a few properties in this file (and distribute it + your server cert / truststores to the NN and each DN in your cluster, and then restart HDFS).

There'a also some more detailed explanation available, here's some links that Google turned up:

• http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.5.0/bk_reference/content/ch_wire1.html

Note that Kerberos will not give you 'secure' communications (if by secure you mean encrypted), Kerberos just provides an authentication mechanism.