So after some debugging the solution wasn't too painful:
- make sure that your rewrite rules that usually rewrite all non-asset URLs to your index file also stop rewriting any requests to the sub application.
- make sure that all asset requests (such as linking to the CSS and JS from your index file) are prefixed with the sub application directory name.
- make sure that all your routes in the sub application are not prefixed with the sub application directory name.
And then it works!