Create JUnit test for a method covering all its invocations in Eclipse workspace

Question

Lets assume following Java class:

class MyClass {
   public String methodA( String data, String expression ) {
      // expression is evaluated using data
      // ...
      // this leads to a result which is returned
      return result;
   }
}

Note, that expression is an instance of an expression language (DSL). Thus, the evaluation of expression using data depends on the combination of both. Normally, expression is a fixed value which does not change often and data can change throughout all invocations.

Well, sometime later a bug is found in MyClass.methodA(String,String). The bug resides in some underlying class and only occurs for some special combination of expression and data. A JUnit test is easily written for this special case and can be fixed.

Unfortunately, this method is used frequently across the whole project. Eclipse call hierarchy identifies more than 97 other methods where this method is used. I now fear regression, if I just apply the bugfix. To feel more safe, I would like to do some regression testing.

Normally, unit tests should consider all important types of invocations, especially border cases. But, as expression is a DSL which can vary highly, it's not easy to test-cover all potential usage. Additionally, those tests would not identify false usage relying on the bug.

So my idea is to proceed in the following way:

1. Find all invocations of this method (like using "Call hierarchy" in Eclipse) and extract all the values for expression.

2. Sample enough real-life values for data (e.g. from database) and cross-evaluate all expressions from the first step on it using the original version of MyClass.methodA(String,String). Save the triples (data, expression, result) to a file.

3. Implement bugfix.

4. Test method MyClass.methodA(String,String) using the above file to assert that the results have not changed.

Following questions:

What do you think of this approach?

Using call hierarchy in Eclipse I can only copy and paste the calling methods but not the exact invocation including arguments to clipboard (cf. step 1). I would have to copy the invoking call by hand for each method found. How do I extract the invocations in a convenient way (in the complete Eclipse workspace, thus in several projects)?

IMHO I am only testing one single method, thus the tests covers only one unit. Is it ok to use JUnit for step 4 or is there something more sophisticated?

Answer 1

As the expected additional value by Testing your software is to cover the most circumstances with lowest costs and maximum outcome, I would agree with your approach.

Collecting reallife samplevalues from your software and save them to a representative file should not be that complex to realize and seems to me as the best way to analyze this thing. Even if you have to copy it manually.