Set Hadoop Job name not in JobConf

Question

I'm writing a small web interface to allow users to submit YARN jobs. I want to give them the possibility to name their jobs to observe their progress later.

The used jar will always be the same, it will just operate on different input data. Currently I'm setting the job name to a constant value in my run() method.

What I'm looking for is something like hadoop jar myjar.jar --jobName=MyJob *myParameters

Is this possible?

Answer 1

Have you tried the simple

-Dmapred.job.name=your_job_name

command line option?

Of course, it means you use the ToolRunner so your main function looks like:

public static void main(String[] args) throws Exception {
    int exitCode = ToolRunner.run(new Driver(), args);
    System.exit(exitCode);
}

and add something like that:

Configuration conf = getConf();
Job job = new Job(conf, conf.get("job.name"));

to create your job in your run function

Answer 2

If you are using mrv2 and have created the job you can use setJobName()

Job job = Job.getInstance(configuration);
job.setJobName("the job name");

Answer 3

ex with sqoop sqoop import \ -Dmapred.job.name='sqoop sequencefile' \ --connect jdbc:mysql://ms.itversity.com:3306/retail_db \ --username retail_user \ --password itversity \ --table orders --target-dir /user/mamatucci/sqoop/orders/as-sequencefile \ -m 2 \ --delete-target-dir \ --as-sequencefile