How to import csv files from ftp server to hadoop(HDFS/HIVE/HBASE)?

StackOverflow https://stackoverflow.com/questions/23387222

  •  12-07-2023
  •  | 
  •  

Question

My source files are located in ftp server in csv format. I want to import those files to HDFS for further processing. I am not aware about how to get those csv files. Can anyone help me on this?

Was it helpful?

Solution

You can make this a 2 step process and upload files to hdfs.

Step 1: Download the files from ftp server and store in local . This should help : Transfer files from FTP server to local unix server

Step 2: Once you have the files in local you can use hadoop fs -put to transfer the files to hdfs.

Example. if you file name is temp.csv you can do:

hadoop fs -put temp.csv $hadoop_path
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top