What's really happening with new and override under the covers?

Question

I've found loads of practical examples of this, and understand the practical output when overriding or hiding methods, but I'm looking for some under the covers info on why this is and why C# allows it when according to the rules of polymorphism, this shouldn't be allowed - at least, insofar as my understanding of polymorphism goes (which seems to coincide with the standard definitions found on Wikipedia/Webopedia).

Class Base
{
    public virtual void PrintName()
    {
        Console.WriteLine("BaseClass");
    }
}

Class FirstDerived : Base
{
    public override void PrintName()
    {
        Console.WriteLine("FirstDerived");
    }
}

Class SecondDerived : Base
{
    public new void PrintName()
    {
        Console.WriteLine("SecondDerived");
    }
}

Using the following code:

FirstDerived b = new FirstDerived();
BaseClass a = b;
b.PrintName();
a.PrintName();

I get:

FirstDerived FirstDerived

Okay, I get that, makes sense.

SecondDerived c = new SecondDerived();
BaseClass a = c;
c.PrintName();
a.PrintName();

I get:

SecondDerived 
BaseClass

Okay, that makes sense, too, instance a can't see c.PrintName() so it's using its own method to print its own name, however I can cast my instance to its true type using:

((SecondDerived)a).PrintName();

or

(a as SecondDerived).PrintName();

to get the output I would expect:

SecondDerived

So what is going on under the covers and what does this mean in terms of polymorphism? I'm told that this facility "breaks polymorphism" - and I guess according to the definition, it does. Is that right? Would an "object oriented" langage like C# really allow you to break one of the core principles of OOP?

Answer 1

(This answers the "why is it allowed" which I think is really the central point of your question. How it works in terms of the IL is less interesting to my mind... let me know if you want me to go into that though. Basically it's just a case of specifying the method to call with a different type token.)

It allows base classes to evolve without breaking derived classes.

Suppose Base didn't originally have the PrintName method. The only way to get at SecondDerived.PrintName would be to have an expression with a static type of SecondDerived, and call it on that. You ship your product, everything is fine.

Now fast forward to Base introducing a PrintName method. This may or may not have the same semantics of SecondDerived.PrintName - it's safest to assume that it doesn't.

Any callers of Base.PrintName know that they're calling the new method - they couldn't have called it before. Any callers which were previously using SecondDerived.PrintName still want to use it though - they don't want to suddenly end up calling Base.PrintName which could do something entirely different.

The difficulty is new callers of SecondDerived.PrintName, who may or may not appreciate that this isn't an override of Base.PrintName. They may be able to notice this from the documentation of course, but it may not be obvious. However, at least we haven't broken existing code.

When SecondDerived is recompiled though, the authors will be made aware that there's now a Base.PrintName class through a warning. They can either stick to their existing non-virtual scheme by adding the new modifier, or make it override the Base.PrintName method. Until they make that decision, they'll keep getting a warning.

Versioning and compatibility isn't usually mentioned in OO theory in my experience, but C# has been designed to try to avoid compatibility nightmares. It doesn't solve the problem completely, but it does a pretty good job.

Answer 2

I answer "how" it works. Jon has answered the "Why" part.

Calls to virtual methods are resolved a bit differently to those of non-virtual ones. Basically, a virtual method declaration introduces a "virtual method slot" in the base class. The slot will hold a pointer to the actual method definition (and the contents will point to an overridden version in the derived classes and no new slot will be created). When the compiler generates code for a virtual method call, it uses the callvirt IL instruction, specifying the method slot to call. The runtime will dispatch the call to the appropriate method. On the other hand, a non-virtual method is called with a call IL instruction, which will be statically resolved to the actual method by the compiler, at compile time (only with the knowledge of the compile-time type of the variable). new modifier does nothing in the compiled code. It essentially tells the C# compiler "Dude, shut up! I'm sure I'm doing the right thing" and turns off the compiler warning.

A new method (actually, any method without an override modifier) will introduce a completely separate chain of methods (new method slot). Note that a new method can be virtual itself. The compiler will look at the static type of the variable when it wants to resolve the method chain and the run time will choose the actual method in that specific chain.

Answer 3

According to the Wikipedia definition:

Type polymorphism in object-oriented programming is the ability of one type, A, to appear as and be used like another type, B

Later on the same page:

Method overriding is where a subclass replaces the implementation of one or more of its parent's methods. Neither method overloading nor method overriding are by themselves implementations of polymorphism.

The fact that SecondDerived does not provide an override for the PrintName does not affect its ability to appear and be used as Base. The new method implementation it provides will not be used anywhere an instance of SecondDerived is treated as an instance of the Base; it will be used only when that instance is explicitly used as an instance of SecondDerived.

Moreover, SecondClass can actually explicitly implement the Base.PrintName in addition to the new hiding implementation, thus providing its own override that will be used when treated as Base. (Though, Base has to be an explicit interface definition or has to derive from one to allow this)