hadoop - Apache Kylin 找不到 HBase 公共库

Question

我已经安装了 Hadoop 2.6.0 版、HBase 0.99.0 版、Hive 1.2 版、Kylin 1.5.0 版。

我已经在独立模式下设置了上述所有内容，而在运行 Kylin 时，它会在早期检查 Hadoop、HBase 和 Hive。一切都已安装，但是当我启动 Kylin 时，它给出了 HBase common lib not found 的错误。以下是 Apache Kylin 的日志。

KYLIN_HOME is set to bin/../
16/03/24 18:02:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
kylin.security.profile is set to testing
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/kunalgupta/Downloads/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/kunalgupta/Downloads/spark-1.6.0-bin-hadoop2.6/lib/spark-assembly-1.6.0-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties
cut: illegal option -- -
usage: cut -b list [-n] [file ...]
       cut -c list [file ...]
       cut -f list [-s] [-d delim] [file ...]
HIVE_CONF is set to: /Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/apache-hive-1.2.1-bin/conf/, use it to locate hive configurations.
HCAT_HOME is set to: /Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/apache-hive-1.2.1-bin/hcatalog, use it to find hcatalog path:
usage: dirname path
find: -printf: unknown primary or operator
hive dependency: /Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/apache-hive-1.2.1-bin/conf/::/Users/kunalgupta/Desktop/kunal/Desktop/Backup/Kunal/Downloads/apache-hive-1.2.1-bin/hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1.jar
cut: illegal option -- -
usage: cut -b list [-n] [file ...]
       cut -c list [file ...]
       cut -f list [-s] [-d delim] [file ...]
hbase-common lib not found

请有人帮帮我。

score 2 · Accepted Answer

您遇到的问题是mac-osx 上的cut命令不支持“--output-delimiter”选项。我在安装 kylin-1.5.1 时遇到了同样的错误。解决这个问题的简单方法是在你的 shell 中使用 gnu 二进制文件而不是 osx 二进制文件。

使用brew安装 coreutils（我将所有常用的 shell utils 更改为它们的 gnu 版本）
为此使用以下命令。

brew install coreutils findutils gnu-tar gnu-sed gawk gnutls gnu-indent gnu-getopt --default-names

现在要让 shell 使用这些而不是 mac 二进制文件，请在 shell 配置文件中添加这些 utils 的路径。

vi ~/.profile

将以下行添加到此文件

PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"

在此之后打开一个新的终端窗口并执行

echo $PATH

结果应该有我们在前面步骤中设置的路径（前置）现在启动 kylin，应该可以顺利运行。

一些对我有帮助的参考链接：
Mac 论坛链接
 Apple.se 的安装指南

score 1 · Accepted Answer

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera/
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$CLASSPATH
export KYLIN_HOME=/usr/local/kylin
export CDH_HOME=/opt/cloudera/parcels/CDH
export SPARK_HOME=${CDH_HOME}/lib/spark
export HBASE_HOME=${CDH_HOME}/lib/hbase
export HIVE_HOME=${CDH_HOME}/lib/hive
export HADOOP_HOME=${CDH_HOME}/lib/hadoop
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

source /etc/profile

将 ${HBASE_HOME} 添加到 /etc/profile

猫 /opt/cloudera/parcels/CDH/lib/hbase/bin/hbase

if [ -n "${shaded_jar}" ] && [ -f "${shaded_jar}" ]; then
  CLASSPATH="${CLASSPATH}:${shaded_jar}"
# fall through to grabbing all the lib jars and hope we're in the omnibus tarball
#
# N.B. shell specifically can't rely on the shaded artifacts because RSGroups is only
# available as non-shaded
#
# N.B. pe and ltt can't easily rely on shaded artifacts because they live in hbase-mapreduce:test-jar
# and need some other jars that haven't been relocated. Currently enumerating that list
# is too hard to be worth it.
#
else
  for f in $HBASE_HOME/lib/*.jar; do
    CLASSPATH=${CLASSPATH}:$f;
  done
  # make it easier to check for shaded/not later on.
  shaded_jar=""
fi
for f in "${HBASE_HOME}"/lib/client-facing-thirdparty/*.jar; do
  if [[ ! "${f}" =~ ^.*/htrace-core-3.*\.jar$ ]] && \
     [ "${f}" != "htrace-core.jar$" ] && \
     [[ ! "${f}" =~ ^.*/slf4j-log4j.*$ ]]; then
    CLASSPATH="${CLASSPATH}:${f}"
  fi
done

score 1 · Accepted Answer

我使用 centos7 ，apache-kylin-3.0.0-alpha2-bin-hadoop3 和 hbase 2.1.4 。我在这些链接中找到了解决方案。 https://issues.apache.org/jira/browse/KYLIN-3564 http://92072234.wiz03.com/share/s/2i1O8Q1L1k042IDoOy3h7BgH2K4G6J2SoQv42Xc4b01xpCrj 第二个链接解决了我的问题。编辑 hbase 文件

$HBASE_HOME/bin/hbase

改变

CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar

至

CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar:/YOUR HBASE FULL PATH or $HBASE_HOME/lib/*

score 0 · Accepted Answer

我解决了它如下：

1. export HBASE_CLASSPATH=/opt/cloudera/parcels/CDH-6.2.0-/hbase-common-2.0.0-cdh6.2.0.jar
2. then start kylin again.

score -1 · Accepted Answer

-1

你在 Windows 上运行吗？抱歉 Kylin 仅在 Linux 1.5 版本上运行

于 2016-03-26T13:11:58.817 回答

hadoop - Apache Kylin 找不到 HBase 公共库

5 回答 5

Related

Reference