There is nothing in MDM protocol which prevent a user from installing and running an app.
The only thing which you can do is to get a list of installed appellation through MDM and do some action (as example alerting or compliance enforcement) if you detected blacklisted app.