Create whole directory structure with Talend DI

Question

I am looking for a way to create a folder struture with Talend DI (with Talend Studio 5.2.0).

I have a temporary folder containing something like this:

/tmp/myjob/type1/file01.txt
/tmp/myjob/type1/file02.txt
/tmp/myjob/type1/file03.txt
/tmp/myjob/type1/fileN.txt
/tmp/myjob/type2/file01.csv
/tmp/myjob/type2/file02.csv
/tmp/myjob/type2/file03.csv
/tmp/myjob/type2/fileN.csv
/tmp/myjob/type3/file01.jpg
/tmp/myjob/type3/file02.jpg
/tmp/myjob/type3/file03.jpg
/tmp/myjob/type3/fileN.jpg

I want to transfer the whole directory structure to a remote FTP (ftps) folder.

And the result would be:

/destination/type1/file01.txt
/destination/type1/file02.txt
/destination/type1/file03.txt
/destination/type1/fileN.txt
/destination/type2/file01.csv
/destination/type2/file02.csv
/destination/type2/file03.csv
/destination/type2/fileN.csv
/destination/type3/file01.jpg
/destination/type3/file02.jpg
/destination/type3/file03.jpg
/destination/type3/fileN.jpg

The idea is to configure a tFTPPut in order to take the /tmp/myjob as root source folder and the talend component to recreate the whole directory structure in the destination (remote) folder.

I tried to put the following lines in the filemask property:

FileMask                newName
"./type1/*.txt"             ""
"./type1/*.csv"             ""
"./type1/*.jpg"             ""

When I execute the job, it make the following error:

[statistics] connected
Exception in component tFTPPut_1
2: No such file

Anybody knows if it is possible to create such a task?

Answer 1

Unfortunately, tFtpPut doesn't give you the possibility to create remote folders. The only workaround you have is to connect via ssh and trig the proper CMD/Bash commands.

About your file mask problem, my advice is to use Perl5 regex syntax to express mask patterns. It's way more powerful, standard and well documented. But if your source structure is quite complex, a better time saving solution could be zipping the whole stuff on the source machine, then ftp the package, finally unzip it (again, via ssh connection) on remote machine.

I know it's not a general solution: as a matter of facts, it's not always possible to ssh on a remote machine. But to my knownledge, it's the only workaround possible atm.