How to ignore multiple failed junit test programmatically?

Question

This is a common-enough problem ("Conditionally ignoring tests in JUnit 4") that JUnit created an Assume class, which is like Assert but causes an AssumptionFailedException that JUnit treats as an ignored test. However, for your case, you probably want to conditionally ignore a lot of tests or test cases.

You'll need some kind of persistence and settings control, first of all. It's nice to ignore the tests on the second run, but would they be re-included on the third, or are they permanently ignored until you manually reset them? How do you reset them?

Now, as for ignoring the test directly: You could create a custom Rule, which decorates a Statement and also receives a Description that contains the method name etc. (You can think of a Statement as a Runnable that often represents a single test method and its decorations.) Firing an AssumptionViolatedException in Rule will probably get you where you need to go, simply, but you'd need to figure out some other way to identify which test is being run.

You may want to create a custom Runner, instead, which gives you a FrameworkMethod that contains everything you could possibly need to identify the test. In that case, BlockJUnit4TestRunner is a good starting point designed for subclassing, especially in its runChild method:

@Override
protected void runChild(final FrameworkMethod method, RunNotifier notifier) {
    Description description = describeChild(method);
    if (isIgnored(method)) {
        notifier.fireTestIgnored(description);
    } else {
        runLeaf(methodBlock(method), description, notifier);
    }
}

... which is overridable, and delegates to runLeaf in ParentRunner. Replace that method with one that checks against your storage to conditionally ignore the test, and you're probably set. Bear in mind, though, that runLeaf is final, and that you'll need to call something else if you want to update your storage when the test fails.