HTML documents must fit HTML specification. XML documents must fit XML specification. XHTML documents must fit XML and HTML specification.
So for XHTML docments you must take the intersection of valid characters from both specifications. Since XHTML is based on XML the specification it cannot allow characters that are forbidden in XML. Therefore the allowed characters in XHTML are always a subset of the ones from XML. So the allowed characters are that from the XHTML specification.