Create EMR Cluster with No Public IP Addresses

Question

I wish to create an EMR cluster where none of the instances are assigned a public IP address for security reasons. I have been able to launch the cluster in my VPC, and using my own custom security group, but for some reason all the nodes are assigned a public IP address by default.

I cant find anything in the EMR CLI documentation about how to disable this: http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-cli-commands.html

Any ideas? Is there some EMR specific reason why the instances MUST have a public IP?

Answer 1

Currently EIPs are needed to access S3 through the Internet Gateway, see: http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-plan-vpc-subnet.html

Completely private clusters are not yet supported.

Answer 2

EMR now supports cluster in private subnet. Reference: https://blogs.aws.amazon.com/bigdata/post/Tx349CL210VDYQF/Securely-Access-Web-Interfaces-on-Amazon-EMR-Launched-in-a-Private-Subnet