java.lang.RuntimeException:Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Question

I have configured my Hive as given on link: http://www.youtube.com/watch?v=Dqo1ahdBK_A, but I am getting the following error while creating a table in Hive. I am using hadoop-1.2.1 and hive-0.12.0.

hive> create table employee(emp_id int,name string,salary double);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient

Answer 1

Looks like problem with your metastore. If you are using the default hive metastore embedded derby. Lock file would be there in case of abnormal exit. if you remove that lock file this issue would be solved

rm   metastore_db/*.lck

Answer 2

This could be an issue with the metastore like sachinjose described or a connection issue. Run hive console in debug mode like so:

hive -hiveconf hive.root.logger=DEBUG,console

Then execute a simple query like show tables; and see what happens. I ran into this issue after restarting a namenode and it was due to the wrong host being set in the configuration files (ec2 generates new private IP/hostname when restarted).

Answer 3

Run this command from the console:

./hive -hiveconf hive.root.logger=DEBUG,console

Now run

show databases;

If you see an exception as below:

java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.

It means there is some permissions issue with the metastore_db. You need to change the permissions of the complete folder. I am running my hive as hdfs user. So, i used the command

chown -R hdfs:hadoop * 

in the hive directory. This resolved the issue.

Answer 4

hive> show databases;
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

To resolve this issue start the hadoop services first.

$ start-all.sh

Then I run

hive> show database;

It works fine for me.

Answer 5

The answer is located in http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH5/5.0/CDH5-Installation-Guide/cdh5ig_hive_schema_tool.html

To suppress the schema check and allow the metastore to implicitly modify the schema, you need to set the hive.metastore.schema.verification configuration property to false in hive-site.xml.

Answer 6

rm metastore_db/*.lck

It works for me, too. It can be found in your home/user directory. You can use the locate command to find it: locate metastore_db

After removing the lock files, close the current session. Call hive shell in new session

Answer 7

make hive.metastore.schema.verification=false in hive-site.xml file this error will be removed. In my case it is working well.

Answer 8

I was getting the same error but I could not even start the hive shell. After trying the very useful hive debug command:

hive -hiveconf hive.root.logger=DEBUG,console

I was able to see that Hive could not find a valid Kerberos TGT. You will see in the debug info something about SASL negotiation failed and no valid Kerberos TGT. My solution was to run

kinit

before running the hive CLI.

Answer 9

Restarting the virtual machine or system should also release the lock.

Answer 10

this could due to more than one "metastore_db". Remove "metastore_db", restart hadoop cluster and open hive shell from $HIVE_HOME/bin folder

Answer 11

Being a newbie i got the same set of errors.
It was found that one of the daemons in my case namenode was not initiated. On installing hadoop it would be wise to make it a habit to hit he following commands :
ps -ef | grep "namenode"
ps -ef | grep "datanode"
ps -ef | grep "tracker"

One needs to check the relevant log if any of the daemon is not working.

Answer 12

If it is in local machine, looks like you have another terminal opened with hive shell/session. You can have only one session using embedded derby data base. Close all other hive sessions and try.

Answer 13

delete code in hive-site.xml and paste this code in hive-site.xml.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=/home/hadoop/Desktop/apache-hive-2.1.1-bin/metastore_db;create=true</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<property>
<name>hive.metastore.uris</name>
<value/>
<description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.PersistenceManagerFactoryClass</name>
<value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value>
<description>class implementing the jdo persistence</description>
</property>
</configuration>

Answer 14

I was facing the same issue there are some important point's that could be resolve this problem.

1.Put the following at the beginning of hive-site.xml

  <property>
    <name>system:java.io.tmpdir</name>
    <value>/tmp/hive/java</value>
  </property>
  <property>
    <name>system:user.name</name>
    <value>${user.name}</value>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:,databaseName=$HIVE_HOME/metastore_db;create=true</value>
    <description>JDBC connect string for a JDBC metastore </description>
  </property>

this is set relative path in absolute URI and configuring Metastore where the database is stored

2.Remove $HIVE_HOME/metastore_db But Be aware, this will remove your schema completely!

3.Now,You must Initialize Derby database.By default, Hive uses Derby database

$HIVE_HOME/bin/schematool -initSchema -dbType derby

also I suppose that your environment variable has been set correctly ,if not please check them that are looking like below:

export HADOOP_HOME=/usr/local/hadoop 
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HIVE_HOME=/usr/lib/hive
export PATH=$PATH:$HIVE_HOME/bin

and then run hive command and type show databases;

Answer 15

I'm surprised to see no one has pointed out the actual issue which is most often the case.

System.setSecurityManager(null)

Put this line at the start of your application.

Answer 16

When I used jdk 11, I deploymented hive on my master node and then those exceptions were thrown. Many methods I have tried but useless. Eventually I changed the version of jdk from 11 to 8, which used in master node. Then I started the hive successfully.

Answer 17

1. remove metastore_db/*.lck
2. restart your Virtual Machine (CentOS, or some other)
3. start the mysql sevice: service mysqld start
4. test: hive -e 'show tables;'

Note that the mysql-connector-xxx-bin.jar has been stored in hive/lib/ and the xml file hive-site.xml has been configurated correctly. Check the username and password about MySQL carefully.

Answer 18

I was facing the same issue--used the below steps to resolve it:

1. Create an file hive-site.xml and input the details (for Local/Prod mode). Make sure the below location exist /home/hadoop/bhishm/warehouse

   Example:

   <?xml version="1.0"?>
   <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
   <configuration>
   <property>
       <name>hive.metastore.warehouse.dir</name>
       <value>/home/hadoop/bhishm/warehouse</value>
       <description>
       Local or HDFS directory where Hive keeps table contents.
       </description>
   </property>
   <property>
       <name>hive.metastore.local</name>
       <value>true</value>
       <description>
       Use false if a production metastore server is used.
       </description>
   </property>
   <property>
       <name>javax.jdo.option.ConnectionURL</name>
       <value>jdbc:derby:;databaseName=/home/hadoop/bhishm/warehouse/metastore_db;create=true</value>
       <description>
       The JDBC connection URL.
       </description>
   </property>
   </configuration>
   

2. Edit the hive-env.sh--> add the java path as the first line after reducing memory usage:

   Example:

   # Hive Client memory usage can be an issue if a large number of clients
   # are running at the same time. The flags below have been useful in 
   # reducing memory usage:
   
   # The java implementation to use.  Required.
   
   export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk
   

3. Run the hive query.