HBase .META. table - readRequestsCount too high

Question

My HBase RegionServers are well balanced except the readRequestsCount to the .META. table.

Looking at the log files of my MapReduce jobs I found that there are many calls to ZooKeeper which established and closed immediately:

2:04,958 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=nam-1:2181,nam-2:2181,dat-4:2181 sessionTimeout=180000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@76477647 2014-02-04 20:32:04,958 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 16972@dat-2 2014-02-04 20:32:04,958 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server /1.10.230.132:2181 2014-02-04 20:32:04,958 WARN org.apache.zookeeper.client.ZooKeeperSaslClient: SecurityException: java.lang.SecurityException: Unable to locate a login configuration occurred when trying to find JAAS configuration. 2014-02-04 20:32:04,958 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 2014-02-04 20:32:04,958 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to nam-1:2181, initiating session 2014-02-04 20:32:04,960 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server nam-1:2181, sessionid = 0x243fd975d060d09, negotiated timeout = 60000 2014-02-04 20:32:05,430 INFO org.apache.zookeeper.ZooKeeper: Session: 0x243fd975d060d09 closed

I cant find any reason for that. Anyone can help?

Answer 1

Can you try setting hbase.zookeeper.property.maxClientCnxns in hbase-site.xml. I set this to 150 it works for me.