My app was approved - and the crash was indeed no issue for the app review team.
I had a chat with Apple about this who were kind enough to call me back and explain the issue. Looks like this phenomenon is "kind of" expected behaviour: Mac Apps may or may not crash when run with a Distribution Profile.
To avoid this problem, we can add both a Development AND a Distribution Profile to our app, without one having to replace the other. This was news to me. Had I however opened my tired eyes a bit wider I would have perhaps spotted the little disclosure triangle myself:
Perhaps this helps those with the same issue.