1

我一直在尝试编写一个 UDAF 来计算加权平均值。我以类似于现有 UDAF 的方式编写它来计算平均值，但它似乎只适用于我的本地机器。我不确定出了什么问题....有人遇到过类似的问题，以下是生成的错误：

java.lang.RuntimeException: Error in configuring object
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:413)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:348)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at                        org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
    at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at     sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
    ... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
    at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
    at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
    at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
    ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
    ... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
    ... 22 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:325)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
    at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:436)
    at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:392)
    at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:166)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:441)
    at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
    at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
    ... 22 more
4

2 回答 2

1

我解决了这个问题..似乎有一些 jar 文件问题......每当我导出一些更改时，有些更改会起作用，而有些则不会，所以我想答案是每次你计划添加时终止你的 hive 会话一个jar文件！：/奇怪，但它的工作...

于 2013-02-21T10:13:40.253 回答
1

在我的情况下，我正在运行 Hive 查询（从使用 serde jar 创建的表中选择语句），但是当我进行选择查询时，类路径中缺少 jar。因此，我使用以下命令将 jar 添加到配置单元中的类路径中：-

添加jar /usr/lib/hive/lib/hive-contrib-0.7.0-cdh3u0.jar；

然后进行选择查询它解决了我的问题。

于 2015-11-18T06:27:13.733 回答