Can't create an external table in hive to point a hbase one

Question

I am a student trying to understand how work the all hadoop thing. So, I'm running cloudera on 15 machines. The configuration is fine, all services are green. I imported a mysql 12k lines under hbase and all went fine too. I wanted to do queries on these data but I know that I can't with hbase. That's why I want to create an external view with the following code:

CREATE EXTERNAL TABLE ViewSimulation2 (
  id int,
  eol int,
  sensor int,
  value1 float,
  value2 float,
  value3 float,
  value4 float,
  value5 float,
  value6 float)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
  "hbase.columns.mapping" =
  ":key,data:eol,data:sensor,data:value1,data:value2,data:value3,data:value4,data:value5,data:value6"
)
TBLPROPERTIES("hbase.table.name" = "Simulation");

When I run it in the console it freezes and I have to ctrl-c to cancel it. In hue, all I have is these messages looping again and again:

13/12/04 07:33:48 INFO ql.Driver: </PERFLOG method=TimeToSubmit start=1386171228376 end=1386171228483 duration=107>
13/12/04 07:33:48 INFO exec.DDLTask: Use StorageHandler-supplied org.apache.hadoop.hive.hbase.HBaseSerDe for table ViewSimulation2
13/12/04 07:33:48 INFO hive.metastore: Trying to connect to metastore with URI thrift://insset-5.cloudera.insset.fr:9083
13/12/04 07:33:48 INFO hive.metastore: Waiting 1 seconds before next connection attempt.
13/12/04 07:33:49 WARN conf.Configuration: fs.default.name is deprecated. Instead, use fs.defaultFS
13/12/04 07:33:49 WARN conf.Configuration: dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
13/12/04 07:33:49 WARN conf.Configuration: io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
13/12/04 07:33:49 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
13/12/04 07:33:49 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 16770@insset-8.cloudera.insset.fr
13/12/04 07:33:49 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:49 INFO util.RetryCounter: Sleeping 2000ms before retry #1...
13/12/04 07:33:51 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:51 INFO util.RetryCounter: Sleeping 4000ms before retry #2...
13/12/04 07:33:56 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:33:56 INFO util.RetryCounter: Sleeping 8000ms before retry #3...
13/12/04 07:34:05 WARN zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
13/12/04 07:34:05 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
13/12/04 07:34:05 WARN zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:344)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1388)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)
13/12/04 07:34:05 ERROR zookeeper.ZooKeeperWatcher: hconnection Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
    at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
    at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
    at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
    at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:684)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:344)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:337)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1388)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

After that, I have other errors like this one :

13/12/04 07:49:53 INFO client.HConnectionManager$HConnectionImplementation: ZooKeeper available but no active master location found
13/12/04 07:49:53 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 8 of 10 failed; retrying after sleep of 16069
org.apache.hadoop.hbase.MasterNotRunningException
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:706)
    at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:126)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73)
    at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:428)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
    at $Proxy13.createTable(Unknown Source)
    at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:576)
    at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3719)
    at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:254)
    at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:138)
    at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:66)
    at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1383)
    at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1169)
    at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute

Which is weird because as I said, all is running fine, so as the hbase master... Would it be a zookeeper bad configuration ? I have made many many searches and I didn't found anything helping me.

Answer 1

It seems like ZooKeeper is not setup correctly. Maybe the dataDir does not exist: http://archive.cloudera.com/cdh4/cdh/4/zookeeper/zookeeperAdmin.html#sc_zkMulitServerSetup