Cannot Load Hive Table into Pig via HCatalog

Question

I am currently configuring a Cloudera HDP dev image using this tutorial on CentOS 6.5, installing the base and then adding the different components as I need them. Currently, I am installing / testing HCatalog using this section of the tutorial linked above.

I have successfully installed the package and am now testing HCatalog integration with Pig with the following script:

A = LOAD 'groups' USING org.apache.hcatalog.pig.HCatLoader(); 
DESCRIBE A;

I have previously created and populated a 'groups' table in Hive before running the command. When I run the script with the command pig -useHCatalog test.pig I get an exception rather than the expected output. Below is the initial part of the stacktrace:

Pig Stack Trace
---------------
ERROR 2245: Cannot get schema from loadFunc org.apache.hcatalog.pig.HCatLoader

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. Cannot get schema from loadFunc org.apache.hcatalog.pig.HCatLoader
    at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1608)
    at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1547)
    at org.apache.pig.PigServer.registerQuery(PigServer.java:518)
    at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:991)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:412)
...

Has anyone encountered this error before? Any help would be much appreciated. I would be happy to provide more information if you need it.

Answer 1

The error was caused by HBase's Thrift server not being proper configured. I installed/configured Thrift and added the following to my hive-xml.site with the proper server information added:

<property>
    <name>hive.metastore.uris</name>
    <value>thrift://<!--URL of Your Server-->:9083</value>
    <description>IP address (or fully-qualified domain name) and port of the metastore host</description>
</property>

I thought the snippet above was not required since I am running Cloudera HDP in pseudo-distributed mode.Turns out, it and HBase Thrift are required to use HCatalog with Pig.