found an answer myself. I had to add the windowSoftInputMode
attribute to make it work:
<activity
android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale"
....
android:windowSoftInputMode="adjustNothing" >