Triggering Pig job with oozie
-
22-07-2021 - |
Question
I have a java code with pig statements embedded inside using the registerQuery()
method of the PigServer
class. Now that I prepare a jar of it, I can run it as hadoop jar jarname
, but the question is how can the same job be triggered using oozie
scheduler. Though it is internally map reduce that works, we cannot call it a map reduce action. Similarly since it is not a pig action, we cannot set that too. Is it possible to run such a task with oozie? What is the best way to schedule such a task?
La solution
Seems like what you are trying to do is not part of Oozie's supported functionality. As your task is neither a typical Java Map Reduce nor a pig action you better off using crontab functionality provided by the operating system to schedule your job or use Spring scheduler.
There is a spring distribution for Hadoop also, called Spring Data - Apache Hadoop which provides a built-in scheduling functionality.