6.1 which points to 3.8 which says:
An identifier is an unlimited-length sequence of Java letters and Java digits, the first of which must be a Java letter.
...
The "Java letters" include uppercase and lowercase ASCII Latin letters A-Z (\u0041-\u005a), and a-z (\u0061-\u007a), and, for historical reasons, the ASCII underscore (_, or \u005f) and dollar sign ($, or \u0024). The $ character should be used only in mechanically generated source code or, rarely, to access pre-existing names on legacy systems.
(emphasis mine)
Though really, it's just a matter of you having a duplicate class name after bytecode generation. The same error would occur with two colliding class names that didn't involve $
in the source.
Edit to add: The binary format is specified in 13.1