Why Hive errors FileNotFoundException when a UDF use some external resource files but runs right at native?

Question

A UDF used some external resource files, then it error: "java.io.FileNotFoundException: resource/placeMap.txt (No such file or directory)", this function runs right at native, why wrong at hive, how can I solved it ? Thank you very much!

Answer 1

look at this. what you want to do is add file <filename> so that <filename> can be added to distributed cache.

it will be accessible as if in the same path as the executable/jar, so check your relatives paths as well.

Answer 2

what hive basicly does is convert your queries to hadoop map-reduce jobs. so they are executed on the machines that have task trackers. so if you are using an external source you eighter need to be sure that that file exists in every single machine at exact same path (and it would be much better to use fullpath) or get the file from hdfs, and load the source file from hdfs. (I prefer using hdfs, much safer that way)