The solution is explicit interface implementation. This essentially makes that method private when you are working with the class that implements the interface. However, it still can be called by casting the instance of the class to the implementing interface, so you still need to throw an exception.
public class Implementation : IInterface
{
void IInterface.SomeMethod()
{
throw new NotSupportedException();
}
}
var instance = new Implementation();
instance.SomeMethod(); // Doesn't compile
var interfaceInstance = (IInterface)instance;
interfaceInstance.SomeMethod(); // Compiles and results in the
// NotSupportedException being thrown