1

我已经在我的机器上安装了 apache-tez-0.8.1-alpha、hadoop 2.7.2 和 hive-2.0.1。我将 mapreduce 框架更改为“yarn-tez”。所以我可以使用 Tez 运行 hadoop 应用程序。但是当我通过在 hive 中设置 hive.execution.engine=tez, 使用 hive 查询运行应用程序。我收到以下错误：

0: jdbc:hive2://192.168.12.252:10000> select count(*) from sample1 where ADD_Length=190;
INFO  : Compiling command(queryId=vasanth_20160613160611_b5ec0f57-6a73-4f13-bd24-ed9f051d120e): select count(*) from sample1 where ADD_Length=190
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:c0, type:bigint, comment:null)], properties:null)
INFO  : Completed compiling command(queryId=vasanth_20160613160611_b5ec0f57-6a73-4f13-bd24-ed9f051d120e); Time taken: 1.336 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=vasanth_20160613160611_b5ec0f57-6a73-4f13-bd24-ed9f051d120e): select count(*) from sample1 where ADD_Length=190
INFO  : Query ID = vasanth_20160613160611_b5ec0f57-6a73-4f13-bd24-ed9f051d120e
INFO  : Total jobs = 1
INFO  : Launching Job 1 out of 1
INFO  : Starting task [Stage-1:MAPRED] in serial mode
INFO  : Tez session hasn't been created yet. Opening session
INFO  : Dag name: select count(*) from samp...ADD_Length=190(Stage-1)
INFO  : 

ERROR : Status: Failed
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1465813808799_0002_1_00, diagnostics=[Vertex vertex_1465813808799_0002_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator
    at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:71)
    at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:139)
    at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:115)
    at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4676)
    at org.apache.tez.dag.app.dag.impl.VertexImpl.access$4300(VertexImpl.java:204)
    at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:3445)
    at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3394)
    at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:3375)
    at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
    at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
    at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
    at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
    at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
    at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1975)
    at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:203)
    at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2090)
    at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2076)
    at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
    at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)
    ... 20 more
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/mapred/split/SplitLocationProvider
    at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.<init>(HiveSplitGenerator.java:96)
    ... 25 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.split.SplitLocationProvider
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 26 more

tez-site.xml：

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
      <name>tez.lib.uris</name>
      <value>${fs.defaultFS}/apps/tez-0.8.1-alpha/tez-0.8.1-alpha.tar.gz</value>
    </property>
 <property>
    <name>tez.am.container.reuse.enabled</name>
    <value>true</value>
    <description>Configuration to specify whether container should be reused</description>
  </property>


</configuration>

我已将 tez-tarball 和 hive-exec-2.0.1.jar 复制到 hdfs 中的 /apps/tez-0.8.1/ 位置。

4

0 回答 0