One of the problems with your approach is that you do not take into consideration what angle the enemy is already facing (his image), he can be at the same (x,y) yet have any of the angles [0,360]. So your rotation seems random because while you rotate him with consideration to the player you do not rotate him with consideration to his own facing. You have to figure out what angle his image is facing, find out what angle the player is facing, then rotate him with regards to this.
I recommend adding a field to every enemy/player Enum facing
(if few and limited angles) or int facing
that holds the angle their image is facing.