How to programmatically Submit Hive Queries using C# in HDInsight Emulator?

Question

I Installed single node HDInsight Emulator in Windows 8 system. I want to programmatically submit hive queries in HDInsight Emulator. Kindly suggest me some ways to submit Hive Queries using C# .

Answer 1

You can pass in BasicAuthCredentials in the latest syntax:

        var creds = new BasicAuthCredential();
        creds.UserName = "hadoop";
        creds.Password = "";
        creds.Server = new Uri("http://localhost:50111");
        var jobClient = JobSubmissionClientFactory.Connect(creds);
        var hiveJob = new HiveJobCreateParameters()
        {
            Query = "select * from hivesampletable limit 10;",
            StatusFolder = "/samplequeryoutput"
        };
        var jobResults = jobClient.CreateHiveJob(hiveJob);

Answer 2

I haven't seen one sample for doing that. But you can a C# .NET SDK sample on submitting Hive jobs at:

Submit Hadoop jobs programmatically http://www.windowsazure.com/en-us/documentation/articles/hdinsight-submit-hadoop-jobs-programmatically/#hive-sdk.

In the following article, you will find how to create the credential object and the URL for the emulator:

Get started using the HDInsight emulator http://www.windowsazure.com/en-us/documentation/articles/hdinsight-get-started-emulator/#powershell

Answer 3

Install Microsoft .Net API for Hadoop WebClient package, this package offers the functionality for submitting jobs to the cluster using REST API:

Install-Package Microsoft.Hadoop.WebClient

Create a WebHCatHttpClient object and provide it with the URL of the cluster, username and password (the following are the defaults):

var client = new WebHCatHttpClient(new Uri("http://localhost:50111"), "hadoop", null);

Submit a Hive job, e.g. list all Hive tables and print them to console:

client.CreateHiveJob("show tables;", null, null, "/queryresult", null);
  .ContinueWith(httpResponseTask => httpResponseTask.Content.ReadAsStringAsync()
     .ContinueWith(outputTask => Console.Out.WriteLine(outputTask.Result)));