Keeping the binary files on the file system will have the least overall performance impact and that's the most appropriate way to go (applications managing tons of binary content like Alfresco use the file system and that works fine for them). If you need your files to be accessed from multiple servers you could store them on a single server and share the directory on the network or just use NAS. However working with the file system is not transactional but if that is critical you may use tools like XADisk that support transactions on a file system level.
JCR is also a good way to go but it is not meant to store only binary files, it's meant to store a "content" (binary data + metadata describing the binary data) in a hierarchical way. However using JCR you are adding another level of abstraction and that may have a performance impact. As the most popular JCR implementations like Apache Jackrabbit, Alfresco and Nuxeo use the file system behind the scene you should consider if you really need the additional features that JCR provides like metadata, versioning, thumbnails, document preview, etc. but with a performance penalty and integration cost or you can just go with the file system if it satisfy your needs.