Sync remote folder with unison

Question

I'm trying to syncrhonize two folders on two different nas. Sadly there is no chance, that the nas can replicate to the other or synchronize with the other nas. By now i synchronize the folder via sftp over a ftp-client which loads the file from one server and uplaods it to the other. Because one of the nas is in the local network, the speed is ok for downloading of the first nas.

For this solution there is one lack. My Laptop or pc needs to be constantly on. Working on this i cam up with the idea to move this process to an raspberry pi.

Currently i can mount both nas with sshfs and it works good. Also i can copy from the first to the second nas. Now i'm working on the sync. Therefore i'm willing to use unison. rsync seems to be designed to replicate all data and not to synchronize (both ways).

The problem i run into is, that using unison is quite slow. this si the config file i use

<folders etc...>
batch = true
auto = true
perms = 0
fastcheck=true
ignoreinodenumbers=true

It seems that the first check for changes needs a bit of time, but after the upload to the remote nas unisons seems to download the file again to check it.

Is there a possibility to fasten the synchronization between two remote folders? Maybe with a additional parameter that i don't have found? Thanks in advance

Answer 1

I have found a solution by myself. Unison doesn't seem to be a solution for the problem.

Now i'm using my raspberry pi and sshfs to mount the two directories of the NAS. Then i wrote a script, which searches for changes between the two folders. After detecting the changes i use wget to copy from one directory to the other. This also has the positive effect that the raspberry pi doesnt load the whole file at once, but only the part which is currently needed for the copy. So it loads with 500 kb/s and puts the file on the remote NAS with 500 kb/s.

Problem solved, thanks anyway

Answer 2

Unison is SLOW for large directories.

Very slow - even on the fastest computers. It doesn't even begin copying, until the entire directory has been scanned and compared for changes with the remote. On my MacBook Pro, it takes over 12 hours to scan the source and destination, before it even begins synchronizing the 2 TB of text and media files.

The scan is especially slow when computing hashes for many or large files. You already use the option to make this faster:

fastcheck = true

If your NAS is using a FAT file system, you may also try:

fat = true

That enables ignoreinodenumbers that you already use, plus a few others, such as case-insensitivity and not calling chmod.