好吧,我不知道这是否会work
在运行时,但您可以通过-jar
选项手动列出 hadoop jar,而不是通过-hadoop
:
./oozie-setup.sh -jars /opt/hadoop/hadoop-1.0.3/hadoop-core-1.0.3.jar:/opt/hadoop/hadoop-1.0.3/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop/hadoop-1.0.3/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop/hadoop-1.0.3/lib/commons-configuration-1.6.jar
如果您查看addtowar.sh
脚本(这是此错误消息的来源),您可以看到一组条件 jar,这些 jar 基于通过-hadoop
选项指定的 hadoop 版本包含在内:
#get the list of hadoop jars that will be injected based on the hadoop version
function getHadoopJars() {
version=$1
if [ "${version}" = "0.20.1" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar"
elif [ "${version}" = "0.20.2" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar"
elif [ "${version}" = "0.20.104" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar"
elif [ "${version}" = "0.20.200" ]; then
#List is separated by ":"
hadoopJars="hadoop-core*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar:commons-configuration-*.jar"
elif [[ "${version}" =~ .*23 ]]; then
suffix="-[0-9.]*"
#List is separated by ":"
hadoopJars="hadoop-mapreduce-client-core${suffix}.jar:hadoop-mapreduce-client-common${suffix}.jar:hadoop-mapreduce-client-jobclient${suffix}.jar:hadoop-mapreduce-client-app${suffix}.jar:hadoop-yarn-common${suffix}.jar:hadoop-yarn-api${suffix}.jar:hadoop-hdfs${suffix}.jar:hadoop-common${suffix}.jar:hadoop-auth${suffix}.jar:guava*.jar:protobuf-*.jar:avro-ipc-*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar:commons-configuration-*.jar"
elif [[ "${version}" =~ 2.* ]]; then
suffix="-[0-9.]*"
#List is separated by ":"
hadoopJars="hadoop-mapreduce-client-core${suffix}.jar:hadoop-mapreduce-client-common${suffix}.jar:hadoop-mapreduce-client-jobclient${suffix}.jar:hadoop-mapreduce-client-app${suffix}.jar:hadoop-yarn-common${suffix}.jar:hadoop-yarn-api${suffix}.jar:hadoop-yarn-client${suffix}.jar:hadoop-hdfs${suffix}.jar:hadoop-common${suffix}.jar:hadoop-auth${suffix}.jar:guava*.jar:protobuf-*.jar:jackson-core-asl-*.jar:jackson-mapper-asl-*.jar:commons-configuration-*.jar:commons-cli-*.jar"
else
echo
echo "Exiting: Unsupported Hadoop version '${hadoopVer}', supported versions: 0.20.1, 0.20.2, 0.20.104, 0.20.200, 0.23.x and 2.x"
echo
cleanUp
exit -1;
fi
}
因此,由于 1.0.x 不在列表中,您可以查看 0.20.104 或 0.20.200 版本并使用 1.0.3 中的相同 jar:
- hadoop-core-1.0.3
- lib/jackson-core-asl-1.8.8.jar
- lib/jackson-mapper-asl-1.8.8.jar
- lib/commons-configuration-1.6.jar
我不明白为什么它不起作用(oozie 只需要与 mapred 和 hdfs 服务进行通信),但您可能在运行时遇到问题 - 值得一试(我已经尝试过 3.1.3-incubating 并且没有我做的少量测试没有任何问题)