If you participate in these projects directly, it looks reasonable to have .spec somewhere in the main source repository. It's just one more file after all, keeping it there requires nothing special from VCS.
If you are to maintain packages for third-party projects, I'd recommend to have separate repo(s) structured like https://github.com/RussianFedora and https://github.com/RussianFedora/unzip This structure just happened to be quite simple and comfortable - you have in one repo your spec file, necessary patches to build for distributions of choice, checksums of upstream sources used.