ModelState is bit bothersome with DateTime because everytime your posted input doesn't match Datetime formatting .isValid() is going to be false.
Have you considered using string to take in user input and later see if its possible to parse into DateTime.
Ps. this also occurs when trying to post and empty string: C# MVC 4 ViewModel not accepting null DateTime
Hope this helps :)