You said it yourself, it happens because the program being compiled is still running.
In Windows, you can't make changes to the exe file of a program when it's running.
It's fantastically annoying, and one of those things that (to me) make Windows slightly less comfortable than e.g. Linux for development.