is it a sufficient protection that it just cannot or can only hardly be extracted out of the bytecode?
"Sufficient" is a subjective term; only you can determine what you feel is sufficient for you.
is it possible to extract the password just from the apk file?
Yes, as APK files can be decompiled, unencrypted network conversations can be sniffed, etc.
how to make it actually less threatening
You can buy a license for DexGuard and use it, as that will encrypt hard-coded strings like your password. Whether that is worth the extra defense is your decision.
would it be better to make the password configurable within an external configuration file
Anyone who roots the device could get at the file.
or generate it randomly during installation of the app (and where should it then be stored)?
It would be stored somewhere that is available to rooted device users, at minimum.