How to create a copy of a directory on Linux with links

Question

I have a series of directories on Linux and each directory contains lots of files and data. The data in those directories are automatically generated, but multiple users will need to perform more analysis on that data and generate more files, change the structure, etc.

Since these data directories are very large, I don't want several people to make a copy of the original data so I'd like to make a copy of the directory and link to the original from the new one. However, I'd like any changes to be kept only in the new directory, and leave the original read only. I'd prefer not to link only specific files that I define because the data in these directories is so varied.

So I'm wondering if there is a way to create a copy of a directory by linking to the original but keeping any changed files in the new directory only.

Answer 1

It turns out this is what I wanted to:

cp -al <origdir> <newdir>

It will copy an entire directory and create hard links to the original files. If the original file is deleted, the copied file still exists, and vice-versa. This will work perfectly, but I found newdir must not already exist. As long as the original files are read-only, you'll be able to create an identical, safe copy of the original directory.

Answer 2

However, since you are looking for a way that people can write back changes, UnionFS is probably what you are looking for. It provides means to combine read-only and read-write locations into one.

Unionfs allows any mix of read-only and read-write branches, as well as insertion and deletion of branches anywhere in the fan-out.

---

Originally I was going to recommend this (I use it a lot):

Assuming the permissions aren't an issue (e.g. only reading is required) I would suggest to bind-mount them into place.

mount -B <original> <new-location>
# or
mount --bind <original> <new-location>

<new-location> must exist as a folder.