You don’t need to do MDM for the enterprise program as far as I know. As for preventing people from downloading the app, I think you just have to keep the link private, or put it behind a URL that can only be accessed on your company network or VPN. And of course, require login, so someone can’t access your internal information just by downloading the app! Presumably, Apple will revoke your enterprise privileges if they find you are abusing them.
Source: I worked at a company that used enterprise distribution for internal beta distribution, among other things.