scp a folder to a remote system keeping the directory layout

Question

I have a large directory tree with hundreds of nested sub-folders. I need to copy only 4 folders and their contents to a remote system, but I need to destination folder structure to be kept the same.

E.G.

./test/sub1/subsub1/hello.txt
./test/sub1/subsub2/hello2.txt    
./test/sub2/hello3.txt

I want to copy ./test/sub1/subsub1/* to a target such as user@system:~/test/sub1/subsub1/* but I do not want to copy subsub2 or sub2.

I have tried using scp as follows:

scp -r ./test/sub1/subsub1 me@my-system:~/test/sub1/subsub1

The result: scp: /test/sub1/subsub1: No such file or directory

I also tried:

scp -r ./test/sub1/subsub1 me@my-system:~/test

This works, but dumps all the files into a single directory. The /test/sub1/subsub1 directory structure is not maintained.

How can I copy a folder, whilst maintaining its structure?

Answer 1

You need a two-pass solution. First, ensure the target directory exists on the remote host:

ssh me@my-system 'mkdir -p ~/test/sub1/subsub1' 

Then, you can copy your files. I recommend using rsync instead of scp, since it's designed for syncing directories. Example usage:

rsync -r -e ssh ./test/sub1/subsub1/ me@my-system:~/test/sub1/subsub1

The -e flag accepts a remote shell to use to carry out the transfer. Trailing slashes are very important with rsync, so make sure yours match the example above.

Answer 2

Use the -R (--relative) option to rsync to preserve the directory structure. Take a look at:

https://serverfault.com/questions/39522/how-to-keep-the-full-path-with-rsync

Here's an example:

[localbox] tree test
test
└── sub1
    └── subsub1
        ├── a
        ├── b
        ├── c
        ├── d
        ├── e
        └── f


 [localbox] rsync -avR test/sub1/subsub1 me@my-system:dest_path
 sending incremental file list
 created directory dest_path
 test/
 test/sub1/
 test/sub1/subsub1/
 test/sub1/subsub1/a
 test/sub1/subsub1/b
 test/sub1/subsub1/c
 test/sub1/subsub1/d
 test/sub1/subsub1/e
 test/sub1/subsub1/f

 sent 406 bytes  received 185 bytes  394.00 bytes/sec
 total size is 0  speedup is 0.00

 [localbox] ssh me@my-system
 [my-system] tree dest_path
 dest_path
 `-- test
    `-- sub1
        `-- subsub1
            |-- a
            |-- b
            |-- c
            |-- d
            |-- e
            `-- f

  3 directories, 6 files

Below is cut from the rsync man page:

-R, --relative

Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames. This is particularly useful when you want to send several different directories at the same time. For example, if you used this command:

     rsync -av /foo/bar/baz.c remote:/tmp/

... this would create a file named baz.c in /tmp/ on the remote machine. If instead you used

     rsync -avR /foo/bar/baz.c remote:/tmp/

then a file named /tmp/foo/bar/baz.c would be created on the remote machine, preserving its full path. These extra path elements are called "implied directories" (i.e. the "foo" and the "foo/bar" directories in the above example).