I tried a few more things based on some suggestion in other forums.
I repaired VS, I reinstalled VS, I tried to restore my default settings. None worked. I tried a simple project to duplicate the issue that I could share to get feedback. I could not get it to not work.
So assuming then the problem was with my project and not VS I made a new project and brought over my forms, modules and classes from project with the problem. I added my references and... edit and continue now works! The only things I changed from the default project were: Target CPU: x86, and Target Framework: .Net Framework 4
I still do not know if there was some corruption or it was a bad setting. This project was originally ported from VB6 many years ago and has been through a few VS upgrades as well.