Question

I want to analyze the Hadoop 1.0.3 Bandwidth consumption in my cluster.
Which tool/s can I use to only analyze the bandwidth consumption between its nodes used by all the processes related with all the hadoop components: hdfs, jobtrackers, tasktrackers, datanode, namenode, heartbeats, data transfer...
Also would like to associate the bandwith consumption with the process.
The main problem is that its processes do not use fix ports. If possible, would it be viable to use a specific ports range or interface/subinterface to use?

Was it helpful?

Solution

There is a whole host of metrics supplied by Ganglia monitoring, when that is activated in your cluster (though I'm not sure if it provides bandwidth consumption).

There is information on it here and here and a nice demo page here.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top