Android documentation for setStreamMute:
Mute or unmute an audio stream.
...
The mute requests for a given stream are cumulative: the AudioManager can receive several mute requests from one or more clients and the stream will be unmuted only when the same number of unmute requests are received.
...
This ended up being the issue. I was calling more mutes than unmutes if there was an error which caused for this locking.