Triggering Pig job with oozie

Question

I have a java code with pig statements embedded inside using the registerQuery() method of the PigServer class. Now that I prepare a jar of it, I can run it as hadoop jar jarname, but the question is how can the same job be triggered using oozie scheduler. Though it is internally map reduce that works, we cannot call it a map reduce action. Similarly since it is not a pig action, we cannot set that too. Is it possible to run such a task with oozie? What is the best way to schedule such a task?

Answer 1

Seems like what you are trying to do is not part of Oozie's supported functionality. As your task is neither a typical Java Map Reduce nor a pig action you better off using crontab functionality provided by the operating system to schedule your job or use Spring scheduler.

There is a spring distribution for Hadoop also, called Spring Data - Apache Hadoop which provides a built-in scheduling functionality.

Answer 2

The Oozie MapReduce action's purpose it to run jobs like 'hadoop jar...'.

The doc details it and some Hue tutorials demo how to run the Oozie job.