0

我正在尝试开始使用 Scio,并且我使用了他们的 giter8 入门项目。https://github.com/spotify/scio.g8

我在 macOS 上使用 Java 8,尝试使用target/pack/bin/word-count --output=wc命令行或命令行运行项目时出现此错误:

sbt "runMain example.WordCount
  --project=xxx
  --input=gs://dataflow-samples/shakespeare/kinglear.txt
  --output=gs://xxx-xxx.appspot.com/wordcount"

堆栈跟踪:

[info] Loading global plugins from /Users/techlead/.sbt/1.0/plugins
[info] Updating ProjectRef(uri("file:/Users/techlead/.sbt/1.0/plugins/"), "global-plugins")...
[info] Done updating.
error: error while loading String, class file '/modules/java.base/java/lang/String.class' is broken
(class java.lang.NullPointerException/null)
[info] Loading settings for project scio-job-build from plugins.sbt ...
[info] Loading project definition from /Users/techlead/Desktop/beamscalaa/scio-job/project
[info] Updating ProjectRef(uri("file:/Users/techlead/Desktop/beamscalaa/scio-job/project/"), "scio-job-build")...
[info] Done updating.
[warn] There may be incompatibilities among your library dependencies; run 'evicted' to see detailed eviction warnings.
error: error while loading String, class file '/modules/java.base/java/lang/String.class' is broken
(class java.lang.NullPointerException/null)
[info] Loading settings for project root from build.sbt ...
[info] Set current project to scio-job (in build file:/Users/techlead/Desktop/beamscalaa/scio-job/)
[info] Compiling 1 Scala source to /Users/techlead/Desktop/beamscalaa/scio-job/target/scala-2.12/classes ...
[info] Done compiling.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/Users/techlead/.sbt/boot/scala-2.12.7/org.scala-sbt/sbt/1.2.7/protobuf-java-3.3.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[info] Packaging /Users/techlead/Desktop/beamscalaa/scio-job/target/scala-2.12/scio-job_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Running example.WordCount --project=xxx --input=gs://dataflow-samples/shakespeare/kinglear.txt --output=gs://xxx-xxx.appspot.com/wordcount
[run-main-0] INFO org.apache.beam.sdk.io.FileBasedSource - Filepattern gs://dataflow-samples/shakespeare/kinglear.txt matched 1 files with total size 157283
[run-main-0] INFO org.apache.beam.sdk.io.FileBasedSource - Splitting filepattern gs://dataflow-samples/shakespeare/kinglear.txt into bundles of size 19660 took 281 ms and produced 1 files and 8 bundles
[error] (run-main-0) org.apache.beam.vendor.guava.v20_0.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.UnsupportedOperationException: Cannot define class using reflection: Cannot define nest member class java.lang.reflect.AccessibleObject$Cache + within different package then class org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.mirror.AccessibleObject
[error] org.apache.beam.vendor.guava.v20_0.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.UnsupportedOperationException: Cannot define class using reflection: Cannot define nest member class java.lang.reflect.AccessibleObject$Cache + within different package then class org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.mirror.AccessibleObject
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2214)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
[error]         at org.apache.beam.runners.direct.DoFnLifecycleManager.get(DoFnLifecycleManager.java:61)
[error]         at org.apache.beam.runners.direct.ParDoEvaluatorFactory.createEvaluator(ParDoEvaluatorFactory.java:124)
[error]         at org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication(ParDoEvaluatorFactory.java:78)
[error]         at org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication(TransformEvaluatorRegistry.java:169)
[error]         at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:117)
[error]         at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error]         at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]         at java.base/java.lang.Thread.run(Thread.java:835)
[error] Caused by: java.lang.UnsupportedOperationException: Cannot define class using reflection: Cannot define nest member class java.lang.reflect.AccessibleObject$Cache + within different package then class org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.mirror.AccessibleObject
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection$Dispatcher$Initializable$Unavailable.defineClass(ClassInjector.java:410)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.loading.ClassInjector$UsingReflection.injectRaw(ClassInjector.java:235)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.loading.ClassInjector$AbstractBase.inject(ClassInjector.java:111)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default$InjectionDispatcher.load(ClassLoadingStrategy.java:232)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.loading.ClassLoadingStrategy$Default.load(ClassLoadingStrategy.java:143)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.TypeResolutionStrategy$Passive.initialize(TypeResolutionStrategy.java:100)
[error]         at org.apache.beam.repackaged.beam_sdks_java_core.net.bytebuddy.dynamic.DynamicType$Default$Unloaded.load(DynamicType.java:5623)
[error]         at org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory.generateInvokerClass(ByteBuddyDoFnInvokerFactory.java:348)
[error]         at org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory.getByteBuddyInvokerConstructor(ByteBuddyDoFnInvokerFactory.java:244)
[error]         at org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory.newByteBuddyInvoker(ByteBuddyDoFnInvokerFactory.java:217)
[error]         at org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory.newByteBuddyInvoker(ByteBuddyDoFnInvokerFactory.java:148)
[error]         at org.apache.beam.sdk.transforms.reflect.DoFnInvokers.invokerFor(DoFnInvokers.java:35)
[error]         at org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load(DoFnLifecycleManager.java:104)
[error]         at org.apache.beam.runners.direct.DoFnLifecycleManager$DeserializingCacheLoader.load(DoFnLifecycleManager.java:91)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.get(LocalCache.java:4053)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4057)
[error]         at org.apache.beam.vendor.guava.v20_0.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4986)
[error]         at org.apache.beam.runners.direct.DoFnLifecycleManager.get(DoFnLifecycleManager.java:61)
[error]         at org.apache.beam.runners.direct.ParDoEvaluatorFactory.createEvaluator(ParDoEvaluatorFactory.java:124)
[error]         at org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication(ParDoEvaluatorFactory.java:78)
[error]         at org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication(TransformEvaluatorRegistry.java:169)
[error]         at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:117)
[error]         at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error]         at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error]         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error]         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error]         at java.base/java.lang.Thread.run(Thread.java:835)
[error] Nonzero exit code: 1
[error] (Compile / runMain) Nonzero exit code: 1
[error] Total time: 13 s, completed 11 mai 2019 à 08:45:44

我也试过java 12,同样的问题。

我做错了什么?

我怎样才能让它工作?

4

1 回答 1

2

Beam/Scio 仅在 0.7.x 版本中为 Java 8

从错误消息看来,您使用的是 Java 11 或更高版本。

https://github.com/raphw/byte-buddy/issues/504

https://github.com/mockito/mockito/issues/1419

于 2019-06-13T15:38:05.380 回答