3

在执行下面给出的脚本时，我收到了如下所述的以下错误。我尝试使用现有目录和非现有目录，但仍然遇到相同的错误。由于我刚刚开始编写猪脚本，请帮助我解决这个问题。应用快速转换后，我正在尝试加载文件并存储在不同的位置。

代码

SET output.compression.enabled true;
SET mapred.map.output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;
A = LOAD '/abc/def/' using PigStorage();
Store A into '/abc/def/snappy/' using PigStorage();

错误

Pig Stack Trace
---------------
ERROR 2116:
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/

org.apache.pig.impl.plan.VisitorException: ERROR 2116:
<file snappy_conversion.pig, line 4, column 0> Output Location Validation Failed for: '/abc/def/snappy/
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75)
        at org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:66)
        at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
        at org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
        at org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
        at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:52)
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:285)
        at org.apache.pig.PigServer.compilePp(PigServer.java:1324)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
        at org.apache.pig.PigServer.execute(PigServer.java:1241)
        at org.apache.pig.PigServer.executeBatch(PigServer.java:335)
        at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:137)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
        at org.apache.pig.Main.run(Main.java:604)
        at org.apache.pig.Main.main(Main.java:157)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.NullPointerException
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at org.apache.pig.builtin.PigStorage.setStoreLocation(PigStorage.java:396)
        at org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:68)
        ... 22 more
================================================================================
4

1 回答 1

1

问题是您设置了错误的配置。正确的设置是

SET output.compression.codec org.apache.hadoop.io.compress.SnappyCodec;

mapred.map.output.compression.codec用于设置映射器输出的中间文件的压缩格式。

您可以在此处阅读代码以获取更多信息：http ://www.grepcode.com/file/repo1.maven.org/maven2/org.apache.pig/pig/0.12.0/org/apache/pig/builtin /PigStorage.java?av=h#440

于 2014-04-08T05:35:19.223 回答