How can i submit HDInsight jobs from nodejs/REST api

Question

I'm looking for a way to submit Pig/Hive jobs from within my node.js application. For this I would like some sort of node.js sdk or REST Api to the service. Is there such a thing.

I did some research and the official npm azure package only handles hdinsight cluster creation, listing and deletion. What I would like is to be able to programmatically submit jobs to the cluster, not by using the power shell (I'm on linux vms) as shown in the http://www.windowsazure.com/en-us/manage/services/hdinsight/submit-hadoop-jobs-programmatically/ tutorial

Thank you!

Answer 1

The HDInsight service has a REST API called WebHCat (formerly Templeton, so you might see that in the docs) which you can use from node easily enough.

The documentation is here http://hive.apache.org/docs/hcat_r0.5.0/rest.html

To just submit a job, the call you're looking for is:

https://mycluster.azurehdinsight.net/templeton/v1/hive

You just post it a query in the execute parameter (form encoded). There are a few other options to do with status location, and completion callbacks which will maybe useful in your application, all documented at http://hive.apache.org/docs/hcat_r0.5.0/hive.html

Note that the templeton instance in HDInsight runs on the standard ssl port (443) rather than the templeton default used in other distributions (50111)