Sharing large media files, which collaboration tools

Question

I am looking for a tool or technique to help share media with a small group of users. Let me first describe roughly what the ideal solution would do for me.

1. Handle large files binary, such as HD home movies and photos.
2. Allow only authenticated users to access the content.
3. Content will be downloaded to the users computer. It will not be streamed or accessed via a cloud. This will make the best use of limited bandwidth.
4. Sychronize content in the background (or via simple command if required) so every user just gets new content as its added.
5. If one user renames a file, alters the directory structure, etc. it is seen by all other users after their next sync.
6. Work with huge data sets on the order of 100's of gigabytes. The only limit should be the disk space available on a users machine.
7. Allow users to synchronize only a subset of the total data set (e.g. only the photos directory, ignoring the home videos folder) if desired. Especially important for those with small hard drives.

Initially I thought of a simple rsync type solution such as CrashPlan which allows you to backup your files to your friends PC. While this meets most of the requirements, namely allow the friend to access the files and having the sync occur in the background, it does not provide a method for said friend to merge new content into the data set. You would have to set up a two way backup. This just becomes unwieldy as the number of users increase.

My next thought was using Git. It has documented problems with large file sizes and I dont really want all prior versions of binary files. During this research I came across Boar. This very interesting package is the closest I have come yet. It handles nearly all the requirements. The only issue I see with Boar is that it is intended to maintain a central repository. If a friend adds content, it gets committed to the server, where everyone else must go to get it. Since I would want each user to have a complete copy of the data set, each user would have to "check out" every file to retrieve it from the Boar server. This is against the Boar design as it is expressely not a distributed version control system. By design you check out what you need, do your work, then check it back in without maintaing the whole repository on your local machine.

So my question for the group is if there is any other tools or techniques I should be considering?

Answer 1

have a look at git annex, it might or might not offer what you need: http://git-annex.branchable.com/