If you don't want the controls to be shown, delete the controls
atribute, but add the autoplay
atribute instead:
<audio loop="loop" autoplay="autoplay">
<source src="NyanCat.mp3" type="audio/mpeg" />
</audio>
Anyway, it's always a good idea to give the user the control to play/stop the music.