Should I be able to find the version of the jar file by just reading a single entry in the manifest.mf-file?
You "may" be able to find the version number that way, but version numbers in JAR files are optional. (Indeed, even the idea of a version is "optional". Nothing in Java requires you to version your code ... though it is undoubtedly a good idea!)
Your "afterthought" idea certainly looks simpler, though you would need to look at the relevant spec to decide if it is required to work.