As per Key-Value Coding Programming Guide :
The validation infrastructure provides a class the opportunity to accept a value, provide an alternate value, or deny the new value for a property and give a reason for the error.
That's it. Other behaviour is not supported out of the box.
It shouldn't be too hard to implement custom behaviour, however:
You could e.g. provide an alternate value in case the validation fails and set some custom flag somewhere in your model that would in turn allow your table view to indicate the failed validation accordingly. Next time the value is validated and passes the validation you could then reset the flag.