The W3C has an article on character encoding that is quite useful. Their take on this is pretty much:
You should always specify the encoding used for an HTML or XML page. If you don't, you risk that characters in your content are incorrectly interpreted. This is not just an issue of human readability, increasingly machines need to understand your data too.
Further, according to the MDN article on the <meta>
element, it is good practice to specify the charset, since it protect your users against certain cross-scripting attacks:
It is good practice, and strongly recommended, to define the character set using this attribute. If no character set is defined for a page, several cross-scripting techniques may become practical to harm the page user, like the UTF-7 fallback cross-scripting technique. Always setting this meta will protect against these risks.
Even though there might be rare cases where it isn't possible to specify the content type, the general opinion appear to be that it is good practice to specify the content type. And if you do so properly, then there isn't much need for using the HTML-representation of special characters (which in my opinion make your code much harder to read).