I confirm this is what git submodules (mentioned by madhead) are for, but you need to realize that a submodule is first a fixed reference to another repo.
You can make modifications directly in a submodule, but you need to:
- commit in that submodule, and pushing to its upstream repo
- cd back to the parent repo
- commit in that parent repo (recording the new reference for the submodule) and pushing to the parent upstream repo
(See more in the "true nature of submodules)
This is well suited for a component-based approach, where each module can evolve independently, and specific revisions of said modules are combined in order to form a complete program.
(your "giant repo" would be a system-based approach, where all modules are writable, and in their latest revision)