Yes, you can use a Java Action. It is, unfortunately, not ideal because Oozie creates a job to then run your ToolRunner
which creates it's own job(s). It does work though.
How can I run MapReduce ToolRunner jobs with Oozie?
Question
I developed a MapReduce application which uses ToolRunner
. Based on the arguments and configuration properties passed by the user the application sets up the job in a certain way: chooses a particular input/output format, chooses the mapper and reducer classes etc.. This logic is implemented in the run(String[] args)
method from my main class, which implements Configured
and Tool
.
Oozie supports MapReduce action nodes, but the problem is that I can't use them with my custom ToolRunner
application. The MapReduce action only allows to define a static job by configuring input/output formats, mapper/reducer classes and key/value classes. My application does this dynamically based on the higher level business logic.
The action should be able to execute something like this command line:
hadoop jar my.jar com.example.MyMainClass -D my.config1=a -D my.config2=b arg1 arg2
Can I create Oozie actions which support ToolRunner
MapReduce applications?
La solution