I wouldn't mix both AngularJS and Symfony apps the same way I wouldn't mix a REST API with all its clients like an Android app, an iOS app, etc. Each has its own structure, idioms and toolchains. So, they all are separate projects stored in separate repositories.
You don't need Symfony to serve an AngularJS app. A web server capable of serving static files — like nginx — is enough.