Scheduler with monitoring features

Question

I have several Java applications that implements ETL helper function. They should run each X minutes. It's good task for cron, but I also need to monitor them - when previous run started, how much time it took, did it fail, when next one will starts etc. Could you advice some scheduling tool with described functionality?

It will be also interesting for me to know what scheduling software is used in your company and what kind of problems does it solve.

UPDATE. Guys, I need standalone application, not scheduling inside existent ones. I don't need to read tons of logs and investigate them - just know that something goes wring with scheduled job.

Answer 1

What you need is a job monitor and that depends a lot on what kind of jobs you're trying to monitor: I suggest you take a look on these options: http://www.quartzdesk.com/ - for quartz jobs http://jenkins-ci.org/ - is actually a ci server, but can be used to monitor tasks http://www.nagios.org/ - for system tasks

Answer 2

In our team, we are using the Quartz Schedulder. Everybody likes it beacause it's simple and scaleable, but I never used it so I can't tell you more about it.

While searching the Internet, I also found a list of Job Schedulders, found here.

Answer 3

I would keep monitoring and scheduling separate. Quartz is good, as another answer suggests.

For monitoring try http://metrics.codahale.com/ or http://jamonapi.sourceforge.net/

Answer 4

If you need to monitor your app in production, try JAMon or AppDynamics (which is payed but has free version). Or better, here is article with links to another java monitoring apps.

For running tasks in cron-like style, Quatrtz scheduler is good solution, it's de-facto standard in this area.

For log monitoring/reading, try Log4j viewer, Logfaces (payed), Otros log viewer, LogMX (payed product). You can find more by search google for "java log viewer" phrase.

But for "when next one will starts" its up to you how you implement it, good solution should be after run of a scheduled task write it in log.

Answer 5

I have been lately working on a similar implementation of Optaplanner It is fabulous, open-source and easily customizable, solves almost all problems of regular scheduling, you can refer its examples as well. You can invoke processes from the java program during the execution cycle of the application as well.

Answer 6

You may be interested in Spring Batch. I'd recommend reading through the introduction chapter of their documentation to get an overview of its features:

http://docs.spring.io/spring-batch/reference/html/spring-batch-intro.html

To summarize: In addition to scheduling jobs, it also covers other aspects such as logging job statistics, transaction management, retry-handling, and splitting up jobs into smaller batches for potentially better performance.

It also covers your requirements of acting as a stand-alone framework for running your jobs, and its exception handling capabilities seem sophisticated.

Answer 7

As a read-to-use scheduling tool, You can check ATrigger. Also analytics, monitoring and exception handling are also available. The REST-API can help you to control tasks in all languages include Java.

Disclaimer: I was among people who made this tool. It is a freeware, not commercial purpose.