My experience suggests the following as best practices:
When a third-party open-source library is used completely as-is, commit a local copy of the downloaded compressed tarball within the main git repo, to avoid network connectivity issues from preventing software builds.
When a third-party open-source library is almost used as-is, but needs to be tweaked (this is common when cross-compiling: many packages need slight adjustments to their configuration steps), store the compressed tarball and a "unified diff" patch file within the main git repo, and apply the patch within the PATCH_COMMAND step of the ExternalProject_Add.
When a third-party open-source library is (to be) heavily modified or extended by your organization, use a separate git repository to hold pointers to the upstream repository (easiest when it also uses git, but upstream svn can be managed also). Commit your organization's changes to a distinct branch from the branch used to mirror upstream. If you wish, you can introduce a submodule relationship between the main git repo and this one, though because the DOWNLOAD_COMMAND can fetch directly from an arbitrary git repo, it is not technically necessary to do so.
It is reasonable to treat small, less-frequently third-party proprietary binaries for a single target platform by archiving them within the main git repo also. However, third-party binaries which are available for a variety of platforms, are large, or frequently evolve should be stored in their own git repo and fetched in via DOWNLOAD_COMMAND, as above.