3

我正在尝试构建一个sbt简单的 hadoop 作业，我正在尝试运行它以尝试在 Amazon EMR 上运行它。但是，当我运行时sbt assembly，出现以下错误：

[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/trenthauck/.ivy2/cache/org.mortbay.jetty/jsp-2.1/jars/jsp-2.1-6.1.14.jar:org/apache/jasper/compiler/Node$ChildInfo.class
[error] /Users/trenthauck/.ivy2/cache/tomcat/jasper-compiler/jars/jasper-compiler-5.5.12.jar:org/apache/jasper/compiler/Node$ChildInfo.class
[error] Total time: 10 s, completed Sep 14, 2013 4:49:24 PM

我试图按照这里的建议https://groups.google.com/forum/#!topic/simple-build-tool/tzkq5TioIqM但是它没有用。

我的 build.sbt 看起来像：

import AssemblyKeys._

mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
  {
    case PathList("org", "apache", "jasper", xs @ _*) => MergeStrategy.last
    case x => old(x)
  }
}

assemblySettings

name := "Scaling Play"

version := "SNAPSHOT-0.1"

scalaVersion := "2.10.1"

libraryDependencies ++= Seq(
  "com.twitter" % "scalding-core_2.10" % "0.8.8",
  "com.twitter" % "scalding-args_2.10" % "0.8.8",
  "com.twitter" % "scalding-date_2.10" % "0.8.8",
  "org.apache.hadoop" % "hadoop-core" % "1.0.0"
  )
4

1 回答 1

4

指令的顺序很重要。您更新程序集设置，以便稍后再次覆盖它。首先定义assemblySettings然后更新它会解决它。

更新的 build.sbt：

import AssemblyKeys._

assemblySettings

mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
  {
    case PathList("org", "apache", "jasper", xs @ _*) => MergeStrategy.last
    case x => old(x)
  }
}

…

之后你会发现有很多冲突的类和其他文件。在这种情况下，您将需要以下合并：

case PathList("org", "apache", xs @ _*) => MergeStrategy.last
case PathList("javax", "servlet", xs @ _*) => MergeStrategy.last
case PathList("com", "esotericsoftware", xs @ _*) => MergeStrategy.last
case PathList("project.clj") => MergeStrategy.last
case PathList("overview.html") => MergeStrategy.last
case x => old(x)

请注意，对类文件使用合并策略可能会出现问题，这是由该特定类的不兼容版本引起的。如果是这种情况，那么您的问题就更大了，因为依赖项彼此不兼容。然后，您必须求助于删除依赖项并找到/制作兼容的版本。

于 2013-09-14T22:36:19.573 回答