What are the environment settings in Apache Pig and Hadoop Connection to run tutorial scripts?

StackOverflow https://stackoverflow.com/questions/4154183

  •  08-10-2019
  •  | 
  •  

Question

I am trying to run the pig tutorial scripts in Ubuntu for two days, however I can not manage to make pig connect to hadoop file system. It is still saying: " Connecting to hadoop file system at: file:///"

In fact, hadoop and pig are both working and I can use them, also pig script under pig installation can connect to the hadoop. "Connecting to hadoop file system at: hdfs://localhost:54310" My hadoop directory is under /home/hadoop/Cluster, pig installation is under /usr/share/pig, pig tutorial is home/hadoop/pig and hadoop installation is /home/someuser/hadoop .

I have tried almost every solution in the web, but I still need some help. Any idea will be appreciated.

Thanks in advance.

Was it helpful?

Solution

You have to start pig as the following:

java -cp /usr/share/pig/pig-0.7.0-core.jar:/home/hadoop/Cluster/conf/ org.apache.pig.Main

Where the /home/hadoop/Cluster/conf/ is the Hadoop configuration directory.

BTW I think it's easier to use Cloudera's Distribution for Hadoop instead of installing your own.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top