0

我正在尝试使用我们使用 Guice 4.0 的内部框架对 HBase 表进行地图缩减

看起来 yarn 可能正在使用 3.0 版,而我遇到了类型不匹配。

2016-03-25 08:19:45,784 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: com.google.inject.multibindings.MultibindingsScanner$1.binder()Lcom/shaded/google/inject/Binder; at com.google.inject.multibindings.MultibindingsScanner$1.configure(MultibindingsScanner.java:53) at com.google.inject.AbstractModule.configure(AbstractModule.java:59) at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223) at com.google.inject.AbstractModule.install(AbstractModule.java:118) at my.packagename.client.KafkaModule.configure(KafkaModule.java:36) at com.google.inject.AbstractModule.configure(AbstractModule.java:59) at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:223) at com.google.inject.spi.Elements.getElements(Elements.java:101) at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:133) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:103) at com.google.inject.Guice.createInjector(Guice.java:95) at com.google.inject.Guice.createInjector(Guice.java:72) at com.google.inject.Guice.createInjector(Guice.java:62) at my.packagename.hbase_fix.CleanerMapper.setup(CleanerMapper.java:82) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

在同一项工作中,我与 fastxml 有类似的不匹配,因此使用maven-shade-plugin来成功重新定位,但我对 com.google.inject 并没有那么成功

<relocations> <relocation> <pattern>com.fasterxml</pattern> <shadedPattern>com.shaded.fasterxml</shadedPattern> <excludes> <exclude>com.fasterxml.*</exclude> </excludes> </relocation> <relocation> <pattern>org.apache.commons.lang</pattern> <shadedPattern>org.shaded.apache.commons.lang</shadedPattern> <excludes> <exclude>org.apache.commons.lang.*</exclude> </excludes> </relocation> <relocation> <pattern>com.google.inject.Binder</pattern> <shadedPattern>com.shaded.google.inject.Binder</shadedPattern> </relocation> <relocation> <pattern>com.google.inject.multibindings</pattern> <shadedPattern>com.shaded.google.inject.multibindings</shadedPattern> </relocation> </relocations>

关于如何在 MapReduce 作业中特别使用 google.inject 获得圆形类型不匹配的任何建议?

4

1 回答 1

0

我不认为有任何简单的解决方案。最后,它归结为强制 mapreduce 类路径首先具有 guice4 的副本,以便代码运行。

不理想,但找到的唯一解决方案

于 2016-04-27T09:38:27.247 回答