parallel execution of mapreduce jobs

StackOverflow https://stackoverflow.com/questions/19293011

  •  30-06-2022
  •  | 
  •  

Question

I have a question. can I execute two or more jobs in hadoop concurrently with JobCntrol? I have 2 jobs that can be execute in parallel and other jobs have dependencies with these two jobs? How can I do that with JobControl? when I use it, it gives the error: java.lang.IllegalStateException: Job in state RUNNING instead of DEFINE

Was it helpful?

Solution

There is no need for a JobControl when you want to execute the jobs in parallel. Just submit the jobs using JobClient.runJob() API. JobControl is used when there is a simple DAG of jobs.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top