For people who would stumble on this page: I did find an answer eventually. You can disable audio ducking by setting the following user defaults:
defaults write com.apple.SpeechRecognitionCore AllowAudioDucking -bool NO
defaults write com.apple.speech.recognition.AppleSpeechRecognition.prefs DictationIMAllowAudioDucking -bool NO
See the detailed explanation on Youtube.