Generating classes automatically from unit tests?

Question

I am looking for a tool that can take a unit test, like

IPerson p = new Person();
p.Name = "Sklivvz";
Assert.AreEqual("Sklivvz", p.Name);

and generate, automatically, the corresponding stub class and interface

interface IPerson         // inferred from IPerson p = new Person();
{
    string Name 
    { 
        get;              // inferred from Assert.AreEqual("Sklivvz", p.Name);
        set;              // inferred from p.Name = "Sklivvz";
    }
}

class Person: IPerson     // inferred from IPerson p = new Person();
{
    private string name;  // inferred from p.Name = "Sklivvz";

    public string Name    // inferred from p.Name = "Sklivvz";
    {
        get
        {
            return name;  // inferred from Assert.AreEqual("Sklivvz", p.Name);
        }
        set
        {
            name = value; // inferred from p.Name = "Sklivvz";
        }
    }

    public Person()       // inferred from IPerson p = new Person();
    {
    }
}

I know ReSharper and Visual Studio do some of these, but I need a complete tool -- command line or whatnot -- that automatically infers what needs to be done. If there is no such tool, how would you write it (e.g. extending ReSharper, from scratch, using which libraries)?

Answer 1

What you appear to need is a parser for your language (Java), and a name and type resolver. ("Symbol table builder").

After parsing the source text, a compiler usually has a name resolver, that tries to record the definition of names and their corresponding types, and a type checker, that verifies that each expression has a valid type.

Normally the name/type resolver complains when it can't find a definition. What you want it to do is to find the "undefined" thing that is causing the problem, and infer a type for it.

For

 IPerson p = new Person();

the name resolver knows that "Person" and "IPerson" aren't defined. If it were

 Foo  p =  new Bar();

there would be no clue that you wanted an interface, just that Foo is some kind of abstract parent of Bar (e.g., a class or an interface). So the decision as which is it must be known to the tool ("whenever you find such a construct, assume Foo is an interface ..."). You could use a heuristic: IFoo and Foo means IFoo should be an interface, and somewhere somebody has to define Foo as a class realizing that interface. Once the tool has made this decision, it would need to update its symbol tables so that it can move on to other statements:

For

 p.Name = "Sklivvz";

given that p must be an Interface (by the previous inference), then Name must be a field member, and it appears its type is String from the assignment.

With that, the statement:

 Assert.AreEqual("Sklivvz", p.Name);

names and types resolve without further issue.

The content of the IFoo and Foo entities is sort of up to you; you didn't have to use get and set but that's personal taste.

This won't work so well when you have multiple entities in the same statement:

 x = p.a + p.b ;

We know a and b are likely fields, but you can't guess what numeric type if indeed they are numeric, or if they are strings (this is legal for strings in Java, dunno about C#). For C++ you don't even know what "+" means; it might be an operator on the Bar class. So what you have to do is collect constraints, e.g., "a is some indefinite number or string", etc. and as the tool collects evidence, it narrows the set of possible constraints. (This works like those word problems: "Joe has seven sons. Jeff is taller than Sam. Harry can't hide behind Sam. ... who is Jeff's twin?" where you have to collect the evidence and remove the impossibilities). You also have to worry about the case where you end up with a contradiction.

You could rule out p.a+p.b case, but then you can't write your unit tests with impunity. There are standard constraint solvers out there if you want impunity. (What a concept).

OK, we have the ideas, now, can this be done in a practical way?

The first part of this requires a parser and a bendable name and type resolver. You need a constraint solver or at least a "defined value flows to undefined value" operation (trivial constraint solver).

Our DMS Software Reengineering Toolkit with its Java Front End could probably do this. DMS is a tool builder's tool, for people that want to build tools that process computer langauges in arbitrary ways. (Think of "computing with program fragments rather than numbers").

DMS provides general purpose parsing machinery, and can build an tree for whatever front end it is given (e.g., Java, and there's a C# front end). The reason I chose Java is that our Java front end has all that name and type resolution machinery, and it is provided in source form so it can be bent. If you stuck to the trivial constraint solver, you could probably bend the Java name resolver to figure out the types. DMS will let you assemble trees that correspond to code fragments, and coalesce them into larger ones; as your tool collected facts for the symbol table, it could build the primitive trees.

Somewhere, you have to decide you are done. How many unit tests the tool have to see before it knows the entire interface? (I guess it eats all the ones you provide?). Once complete, it assembles the fragments for the various members and build an AST for an interface; DMS can use its prettyprinter to convert that AST back into source code like you've shown.

I suggest Java here because our Java front end has name and type resolution. Our C# front end does not. This is a "mere" matter of ambition; somebody has to write one, but that's quite a lot of work (at least it was for Java and I can't imagine C# is really different).

But the idea works fine in principle using DMS.

You could do this with some other infrastructure that gave you access to a parser and an a bendable name and type resolver. That might not be so easy to get for C#; I suspect MS may give you a parser, and access to name and type resolution, but not any way to change that. Maybe Mono is the answer?

You still need a was to generate code fragments and assemble them. You might try to do this by string hacking; my (long) experience with gluing program bits together is that if you do it with strings you eventually make a mess of it. You really want pieces that represent code fragments of known type, that can only be combined in ways the grammar allows; DMS does that thus no mess.

Answer 2

Its amazing how no one really gave anything towards what you were asking.

I dont know the answer, but I will give my thoughts on it.

If I were to attempt to write something like this myself I would probably see about a resharper plugin. The reason I say that is because as you stated, resharper can do it, but in individual steps. So I would write something that went line by line and applied the appropriate resharper creation methods chained together.

Now by no means do I even know how to do this, as I have never built anything for resharper, but that is what I would try to do. It makes logical sense that it could be done.

And if you do write up some code, PLEASE post it, as I could find that usefull as well, being able to generate the entire skeleton in one step. Very useful.

Answer 3

If you plan to write your own implementation I would definately suggest that you take a look at the NVelocity (C#) or Velocity (Java) template engines.

I have used these in a code generator before and have found that they make the job a whole lot easier.

Answer 4

It's doable - at least in theory. What I would do is use something like csparser to parse the unit test (you cannot compile it, unfortunately) and then take it from there. The only problem I can see is that what you are doing is wrong in terms of methodology - it makes more sense to generate unit tests from entity classes (indeed, Visual Studio does precisely this) than doing it the other way around.

Answer 5

I think a real solution to this problem would be a very specialized parser. Since that's not so easy to do, I have a cheaper idea. Unfortunately, you'd have to change the way you write your tests (namely, just the creation of the object):

dynamic p = someFactory.Create("MyNamespace.Person");
p.Name = "Sklivvz";
Assert.AreEqual("Sklivvz", p.Name);

A factory object would be used. If it can find the named object, it will create it and return it (this is the normal test execution). If it doesn't find it, it will create a recording proxy (a DynamicObject) that will record all calls and at the end (maybe on tear down) could emit class files (maybe based on some templates) that reflect what it "saw" being called.

Some disadvantages that I see:

• Need to run the code in "two" modes, which is annoying.
• In order for the proxy to "see" and record calls, they must be executed; so code in a catch block, for example, has to run.
• You have to change the way you create your object under test.
• You have to use dynamic; you'll lose compile-time safety in subsequent runs and it has a performance hit.

The only advantage that I see is that it's a lot cheaper to create than a specialized parser.

Answer 6

I like CodeRush from DevExpress. They have a huge customizable templating engine. And the best for me their is no Dialog boxes. They also have functionality to create methods and interfaces and classes from interface that does not exist.

Answer 7

Try looking at the Pex , A microsoft project on unit testing , which is still under research

research.microsoft.com/en-us/projects/Pex/

Answer 8

I think what you are looking for is a fuzzing tool kit (https://en.wikipedia.org/wiki/Fuzz_testing).

Al tough I never used, you might give Randoop.NET a chance to generate 'unit tests' http://randoop.codeplex.com/

Answer 9

Visual Studio ships with some features that can be helpful for you here:

Generate Method Stub. When you write a call to a method that doesn't exist, you'll get a little smart tag on the method name, which you can use to generate a method stub based on the parameters you're passing.

If you're a keyboard person (I am), then right after typing the close parenthesis, you can do:

• Ctrl-. (to open the smart tag)
• ENTER (to generate the stub)
• F12 (go to definition, to take you to the new method)

The smart tag only appears if the IDE thinks there isn't a method that matches. If you want to generate when the smart tag isn't up, you can go to Edit->Intellisense->Generate Method Stub.

Snippets. Small code templates that makes it easy to generate bits of common code. Some are simple (try "if[TAB][TAB]"). Some are complex ('switch' will generate cases for an enum). You can also write your own. For your case, try "class" and "prop".

See also "How to change “Generate Method Stub” to throw NotImplementedException in VS?" for information snippets in the context of GMS.

autoprops. Remember that properties can be much simpler:

public string Name { get; set; }

create class. In Solution Explorer, RClick on the project name or a subfolder, select Add->Class. Type the name of your new class. Hit ENTER. You'll get a class declaration in the right namespace, etc.

Implement interface. When you want a class to implement an interface, write the interface name part, activate the smart tag, and select either option to generate stubs for the interface members.

These aren't quite the 100% automated solution you're looking for, but I think it's a good mitigation.

Answer 10

I find that whenever I need a code generation tool like this, I am probably writing code that could be made a little bit more generic so I only need to write it once. In your example, those getters and setters don't seem to be adding any value to the code - in fact, it is really just asserting that the getter/setter mechanism in C# works.

I would refrain from writing (or even using) such a tool before understanding what the motivations for writing these kinds of tests are.

BTW, you might want to have a look at NBehave?

Answer 11

I use Rhino Mocks for this, when I just need a simple stub.

http://www.ayende.com/wiki/Rhino+Mocks+-+Stubs.ashx