我正在尝试在我的 linux 机器上启动spark-job-server 。我做了以下事情:
- 安装 cloudera 发行版 CDH(5.x) 并启动并运行
spark-job-server
从上面提到的github下载- 将项目提取到某个文件夹中
- 安装 sbt-0.13.8 并设置路径变量并能够运行 sbt 命令
- 进入项目目录 (
spark-job-server/
) 并运行 sbt 命令 - 在 SBT shell 中,我输入了“re-start”,它应该启动
spark-job-server
.
然后,我得到了这个异常:
:: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[warn] :: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.apache.hadoop:hadoop-mapreduce-client-app:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-yarn-api:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-mapreduce-client-core:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] org.apache.hadoop:hadoop-mapreduce-client-jobclient:2.2.0
[warn] +- org.apache.hadoop:hadoop-client:2.2.0
[warn] +- org.apache.spark:spark-core_2.10:1.3.1 (/home/analytics/data_dir/spark-jobserver-master/project/Build.scala#L186)
[warn] +- spark.jobserver:job-server-api_2.10:0.5.2-SNAPSHOT
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[warn] Credentials file /root/.bintray/.credentials does not exist
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[trace] Stack trace suppressed: run last job-server-api/*:update for the full output.
[trace] Stack trace suppressed: run last akka-app/*:update for the full output.
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (job-server-api/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] (akka-app/*:update) sbt.ResolveException: unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-app;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-yarn-api;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-core;2.2.0: not found
[error] unresolved dependency: org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.2.0: not found
[error] Total time: 5 s, completed Aug 6, 2015 1:29:26 PM
更多信息:
- 我已经手动下载了这些依赖项并放置了它们
- 当我再次重新运行重新启动命令时,sbt 仍然无法找到我在
.ivy2
文件夹中复制的相同依赖项
我怎样才能解决这个问题?