Should separation between API and Implementation be total?

Question

Separation between API design and their implementation is often recommended in large software implementations. But somewhere, they have to be reconnected (i.e., the implementation has to be reconnected to the API).

The following example shows an API design and an invocation of its implementation via the INSTANCE object:

import java.util.List;

public abstract class Separation {

    public static final Separation INSTANCE = new SeparationImpl();

    // Defining a special list
    public static interface MySpecialList<T> extends List<T> {
        void specialAdd(T item);
    }

    // Creation of a special list
    public abstract <T> MySpecialList<T> newSpecialList(Class<T> c);

    // Merging of a special list
    public abstract <T> MySpecialList<? extends T> specialMerge(
        MySpecialList<? super T> a, MySpecialList<? super T> b);

    // Implementation of separation
    public static class SeparationImpl extends Separation {

        @Override
        public <T> MySpecialList<T> newSpecialList(Class<T> c) {
            return ...;
        }

        @Override
        public <T> MySpecialList<? extends T> specialMerge(
            MySpecialList<? super T> a, MySpecialList<? super T> b) {
            return ...;
        }

    }

}

Some will argue that API should not refer to implementation code. Even if we separate API code from implementation via separate files, one often has to import implementation code (at least the class name) in the API.

There are techniques to avoid such references by using a string representation of the fully qualified name. The class is loaded with that string and then instantiated. It makes the code more complicated.

My question: Is there any benefit to completely separate or isolate API code from implementation code? Or is this just a purist's attempt to reach perfection with little practical benefits?

Answer 1

I've always understood the requirement to separate interface from implementation to mean that you don't mix the how of your implementation with the what. So in your above example, mixing api and implementation would mean exposing in the api something that was specific to how SeparationImpl implemented your api.

As an example, look at how iteration is implemented in various collection classes. There are more specific ways you can retrieve elements in specific collections (e.g. by position in an ArrayList) but those are not exposed in Collection because they're specific to how the concrete ArrayList is implemented.

I've also seen projects with huge directories of interfaces, each of which has a single concrete implementation, and each of which mechanically reproduces every method in their concrete implementation, which seems like a completely pointless "pretend" abstraction, as it's not actually providing any logical abstraction.

Answer 2

One technique which is often using in OSGi is to have the API in a separate module to the implementation. The API should compile by itself avoiding any reference to the implementation directly.

Answer 3

Peter's and Steve's answers are enough but I would like to add more - if you ever have only single implementation of the interface or abstract class, then its pointless to have interface or abstract class as its defeats the purpose of abstraction.
In your case I really didn't understand - why you implemented Separation as a abstract class, rather SeparationImpl itself can be API class or if you have different implementations Separation can be an inetrface and if you have some common functionality then you can have another abstract class implementing your interface and then SeparationImpl inheriting from that abstract class. the sample class hierarchy would look like

interface Separation --> AbstractSeparation --> SeparationImpl 

just like the standard collection library

interface List --> AbstractList --> ArrayList

Answer 4

Additional to the good points from the other authors I would mention the unit testing purposes:

To mock up objects is highly easier when having interfaces intstead of classes.