How to import csv files from ftp server to hadoop(HDFS/HIVE/HBASE)?

Question

My source files are located in ftp server in csv format. I want to import those files to HDFS for further processing. I am not aware about how to get those csv files. Can anyone help me on this?

Answer 1

You can make this a 2 step process and upload files to hdfs.

Step 1: Download the files from ftp server and store in local . This should help : Transfer files from FTP server to local unix server

Step 2: Once you have the files in local you can use hadoop fs -put to transfer the files to hdfs.

Example. if you file name is temp.csv you can do:

hadoop fs -put temp.csv $hadoop_path