我在运行地精作业时遇到错误。我的 core-site.xml 看起来不错,并且具有所需的值。
核心站点.xml
<property>
<name>fs.AbstractFileSystem.gs.impl</name>
<value>com.google.cloud.hadoop.fs.gcs.GoogleHadoopFS</value>
<description>The AbstractFileSystem for 'gs:' URIs.</description>
</property>
错误
org.apache.gobblin.runtime.ForkException: Fork branches [0] failed for task task_toGCPHIVE_1639057335724_14
<Fork 0>
java.lang.RuntimeException: Error creating writer
at org.apache.gobblin.writer.PartitionedDataWriter$4.get(PartitionedDataWriter.java:214)
at org.apache.gobblin.writer.PartitionedDataWriter$4.get(PartitionedDataWriter.java:207)
at org.apache.gobblin.writer.CloseOnFlushWriterWrapper.<init>(CloseOnFlushWriterWrapper.java:73)
at org.apache.gobblin.writer.PartitionedDataWriter.<init>(PartitionedDataWriter.java:206)
at org.apache.gobblin.runtime.fork.Fork.buildWriter(Fork.java:562)
at org.apache.gobblin.runtime.fork.Fork.buildWriterIfNotPresent(Fork.java:570)
at org.apache.gobblin.runtime.fork.Fork.processRecord(Fork.java:516)
at org.apache.gobblin.runtime.fork.AsynchronousFork.processRecord(AsynchronousFork.java:103)
at org.apache.gobblin.runtime.fork.AsynchronousFork.processRecords(AsynchronousFork.java:86)
at org.apache.gobblin.runtime.fork.Fork.run(Fork.java:250)
at org.apache.gobblin.util.executors.MDCPropagatingRunnable.run(MDCPropagatingRunnable.java:39)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:111)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:58)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hadoop.fs.UnsupportedFileSystemException: fs.AbstractFileSystem.gs.impl=null: No AbstractFileSystem configured for scheme: gs
at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:160)
我能够在命令行中运行 GS 命令而没有任何问题。例如:hadoop fs -ls gs://<<bucketName>>
产生所需的输出。
任何帮助,将不胜感激 。