roman-mazur answered the question in another issue at Github.
There is currently a discussion going on, whether it's the right way. But for the moment adding these lines solved my issue:
afterEvaluate {
javadocs.classpath += files(android.plugin.runtimeJarList)
}