FAILED: SemanticException [Error 10001]

Question

we created a HDinsight cluster associated with a blob storage account, and then we created hive tables and acceded to it using hive queries without problems. but to remove the cluster and create a new one (associated to same blob storage) , the hive tables are not visible from this. when we run hive queries we have the following error message:

Logging initialized using configuration in file:/C:/apps/dist/hive-0.12.0.2.0.7.0-1559/conf/hive-log4j.properties FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'tableName'

is possible that hive tables created from a cluster run in a another one without problems? or once the tables are created in a cluster do not work in another?

Thank you.

Answer 1

Yes, the Hive table information is stored is the metastore. The metastore is designed to be attached to different HDInsight cluster. Here is a general procedure on getting it to work:

1. create an HDInsight cluster with Hive metastore specified and one or more additional storage accounts for data storage.
2. create external Hive tables pointing to the data stored in those additional storage accounts.
3. delete the cluster.
4. create another HDInsight using the same Hive metastore, and the same additional storage accounts. The Hive tables are visible on the new HDInsight cluster.

You should be able to use Hive internal tables as well. Hive internal tables store the data on the default storage account. When you create the new cluster, you must use the same storage account and the same metastore. I haven't tested the internal hive table scenario myself.

Hope it help.