1

我在猪中创建了以下脚本。我对 PIG 和 PIGLATIN 很陌生。我仍在学习如何有效地使用 PIG 脚本。

执行脚本后出现此错误：

错误 ERROR [main] org.apache.pig.tools.grunt.Grunt - 错误 2997：无法从后端错误重新创建异常：org.apache.pig.backend.executionengine.ExecException

有人可以解释原因以及我如何纠正它。在 csv 文件中，除了具有整数值的 rate 列之外，我有所有 char 列。

*divs = LOAD 'output\file.csv' using PigStorage(',') AS (uniID:chararray, deal:chararray, rol: chararray,name:chararray,add:chararray,city:chararray,stat:chararray,stn:chararray,zip:chararray,country:chararray,db:chararray,sm:chararray,rate:int);
DUMP divs;
trimmed = foreach divs generate sm,uniID,rol,rate,country;
DUMP trimmed;
grpd = group trimmed by sm;
orderd = order trimmed by country;
describe trimmed;
describe grpd;
DUMP grpd;
describe orderd;
avgdiv = foreach grpd generate sm, AVG(divs.rate), SUM(divs.rate), MAX(divs.rate);
DUMP avgdiv;
store avgdiv into 'output/pigdescribe1out';
explain;*
4

1 回答 1

1

您的 group 语句返回错误。您正在尝试在分组之前聚合数据，因此会出现错误。

divs = LOAD '$input' using PigStorage('^A') AS (uniID:chararray, deal:chararray, rol: chararray,name:chararray,add:chararray,city:chararray,stat:chararray,stn:chararray,zip:chararray,country:chararray,db:chararray,sm:chararray,rate:int);<br/>
DUMP divs;
trimmed = foreach divs generate sm,uniID,rol,rate,country;
DUMP trimmed;
grpd = group trimmed by sm;
orderd = order trimmed by country;
describe trimmed;
describe grpd;
DUMP grpd;
describe orderd;
avgdiv = foreach grpd generate FLATTEN(group), AVG(trimmed.rate), SUM(trimmed.rate), MAX(trimmed.rate);
DUMP avgdiv;
store avgdiv into 'data/sampledata/';
explain;

这工作得很好。

于 2013-02-01T15:35:18.213 回答