It seems it needed some time to occur to me that nothing stops me from moving the templates out of the web project and put them into a library project, where there is always a project file and it will be working well regardless of the type of web project, which solves my problem.
I still don't know how the precompiled T4 text templates are supposed be working with a web page type of project though. Feel free to expand on this if you know more.