This is happening because sender
is declared as type id
, which means that the compiler knows it's an Objective-C object, but it doesn't know what class it belongs to. So it looks though all the class definitions and it found two matches, and it doesn't know which one to use.
The solution is to either cast it or use a new variable of the correct type:
BOOL replaceCheckBoxFlag = ([(NSCell *)sender state] == NSOnState);
or
NSCell *cell = sender;
BOOL replaceCheckBoxFlag = ([cell state] == NSOnState);