Pig Output Location Validation Failed

Question

While executing the given below script, i am getting the folloring error as mentioned below. I tried with existing directory as well as non existing directory but i am still getting the same error. As i have just started writing the pig script, please help me out with this. I am trying to load the file and store in different location after applying the snappy conversion.

Code

SET output.compression.enabled true;
SET mapred.map.output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;
A = LOAD '/abc/def/' using PigStorage();
Store A into '/abc/def/snappy/' using PigStorage();

Error

Pig Stack Trace
---------------
ERROR 2116:
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/

org.apache.pig.impl.plan.VisitorException: ERROR 2116:
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75)
        at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66)
        at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
        at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
        at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:285)
        at org.apache.pig.PigServer.compilePp(PigServer.java:1324)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
        at org.apache.pig.PigServer.execute(PigServer.java:1241)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:335)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:604)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.NullPointerException
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at org.apache.pig.builtin.PigStorage.setStoreLocation(PigStorage.java:396)
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:68)
        ... 22 more
================================================================================

Answer 1

The problem is that you set the wrong configuration. The correct setting is

SET output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;

mapred.map.output.compression.codec is used to set the compress format of the intermediate files which are output by mappers.

You can read the codes here to get more information: http://www.grepcode.com/file/repo1.maven.org/maven2/org.apache.pig/pig/0.12.0/org/apache/pig/builtin/PigStorage.java?av=h#440